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PREFACE 


people,  and  should  be  obvious  to  the  research  and  development 
community,  field  data  shows  examples  of  equipment  in  operational 
use  where  reliability  and  maintainability  fall  far  short  of  our 


expectations  and  needs.  Programs  are  underway  in  both  Air  Force 
Systems  Command  and  Air  Force  Logistics  Coinnand  to  improve  those 
equipments,  but  steps  must  be  taken  to  insure  that  deficient 
equipments  are  not  deployed  in  the  future.  As  top  Air  Force  officials 
have  emphasized,  enormous  life  cycle  cost  savings  can  result  from 
improved  reliability  and  maintainability.  Furthermore,  military 
force  limitations  compel  us  to  squeeze  as  much  effectiveness  out  of 


every  piece  of  equipment  as  we  can,  and  improving  reliability  and 
maintainability  is  one  way  to  do  it. 


Some  Reliability  and  Maintainability  (R&M)  problems  can  be  laid  to 
technology  (e.g.,  a jammer  might  require  high  power  that  can  only  be 
provided  with  a new  tube  whose  reliability  problems  have  not  been 
resolved,  and  space  limitations  may  require  packaging  adversely 
impacting  maintainability).  However,  many  R&M  problems  can  be 
avoided  by  astute  management  decisions.  While  we  will  readily  agree 
that  the  technical  problems  cannot  be  ignored,  this  report  is 
concerned  with  R&M  management  on  the  premise  that  better  management 
is  a high-payoff,  low  cost  approach  to  more  reliable  and  maintainable 
equipment.  The  resulting  management  guidelines  are  written  to  fit 
within  the  scope  of  present  regulations. 
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Chapter  1 
INTRODUCTION 

PURPOSE  OF  THIS  GUIDE 

Since  World  War  II,  the  costs  of  buying  and  supporting  new 
systems  and  equipment  have  shown  alarming  growth,  even  after 
discounting  dollar  Inflation.  These  growing  costs  have  been 
fed  by  the  evolution  of  more  sophisticated  weapons  designed 
for  conflict  In  new  domains  of  speed  and  space  and  often  de- 
signed for  nearly  Instant  response.  While  modes  of  conflict 
over  a half  century  old  remain  with  us  as  modern  threats,  we 
have  seen  the  addition  of  globe  girdling  aircraft,  missiles, 
satellites,  coiiinuni cation  systems  and  radars;  and  these  In 
turn  have  spawned  new  families  of  defense  countermeasures  and 
offense  counter-countermeasures.  Furthermore,  the  omen  of 
possible  nuclear  destruction  has  motivated  defense  planners 
and  engineers  to  build  superior  performance  Into  every  weapon 
and  defense  system  and  to  apply  the  latest  technology  to  the 
greatest  extent  possible.  In  this  scene  of  revolutionary  tech- 
nical development  and  International  competition,  enormous  cost 
growth  has  been  unavoidable. 

Costs  continue 
to  rise  and  pro- 
jections for  the 
future  are  sober 
Ing.  At  the 
same  time,  the 
buying  power  of 
the  defense  bud- 
get Is  shrink- 
ing. It  Is  ob- 
vious that  some 
of  our  ways  of 
doing  defense 
business  must 
be  Improved. 

One  change  which 
can  yield  sav- 
ings of  very 
substantial 
proportions 

Is  Improved  reliability  and  maintainability  (R&M)  of  systems  and 
equipments.  Top  defense  leaders  recognize  that  Improved  R&M  have 
the  potential  to  save  billions  of  dollars  over  the  next  decade  by 
avoiding  unnecessary  costs  for  spares,  repairs,  personnel  and  extra 
systems.  At  the  same  time.  Improved  R&M  Increases  the  operational 
effectiveness  of  equipment  by  Improving  Its  chance  of  being  In  com- 
mission at  the  time  It  Is  needed.  The  dual  payoffs  of  reduced  lo- 
gistics costs  and  Improved  operational  effectiveness  can  be  realized 
with  development  cost  Increases  which  are  very  modest  when  compared 
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To  cash  in  on  savings.  Air  Force  managers  of  development  and 
logistic  support  programs  have  been  asked  to  take  bold  steps 
leading  to  more  reliable  and  maintainable  systems  for  our 
operational  people.  These  requests  for  action  have  come  from 
the  Congress,  the  Secretary  of  Defense,  the  Secretary  of  the 
Air  Force,  the  Air  Force  Chief  of  Staff  and  many  commanders 
and  directors.  There  has  been  no  mincing  of  words.  Everyone 
wants  better  R&M, 

In  response  to  requests  for  bold  steps  towards  better  R4M, 
niany  actions  are  underway;  studies  of  management  practice, 
procurement  policy  and  hardware  design;  retrofit  programs; 
data  reporting  improvements;  action  committees;  revised  reg- 
ulations; and  so  on.  The  preparation  and  promotion  of  this 
R&M  management  guidebook  is  another. 

While  the  need  for  reliable  and  mantainable  military  equip- 
ment appears  to  be  obvious,  what  may  not  be  so  obvious  in  a 
specific  project  are  answers  to  these  questions: 

How  much  R&M  do  we  need? 

How  can  we  get  it? 

How  much  does  it  cost? 

What  is  the  payoff  in  the  long  run? 

The  answers  to  these  questions  have  both  technical  and  manage- 
ment aspects.  The  technical  aspects  (mathematical  models,  pre- 
diction methods,  test  procedures)  are  described  injnumerous 
volumes  of  Government  and  commercial  literature  and  are  taught 
in  many  educational  programs.  On  the  other  hand;  management 
aspects  (planning,  organizing,  manning,  leading,  and  controlling) 
are  given  insufficient  attention.  The  purpose  of  this  report  is 
to  provide  guidance  on  these  management  aspects  of  R&M. 

History  shows  Air  Force  projects  with  both  outstanding  success 
and  embarrassing  failure  in  achieving  R&M  goals.  The  procedures 
and  actions  recommended  in  this  report  reflect  the  lessons  learned 
from  the  management  approaches  used  in  both  the  successes  and 
failures.  These  recommendations  are  consistent  with  the  policies 
and  authority  stated  in  present  Air  Force  regulations,  and  are 
applications  of  management  fundamentals  proven  valid  through 
decades  of  use  in  countless  organizations.  The  recommendations 
should  be  easily  understood  by  any  manager  who  has  a general  back- 
ground in  engineering. 
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This  guidebook  should 
be  a useful  reference 
for  every  R&M  manager 
as  well  as  for  the 
System  Program  Direc- 
tor and  his  staff  and 
staff  personnel  at 
other  levels  of  system 
acquisition  management. 
If  It  causes  only  a 
single  major  equipment 
development  program  to 
go  from  marginal  RSH 
achievement  to  complete 
success,  the  logistics 
savings  over  the  equip- 
ment life  cycle  can 
easily  be  tens  of  mil- 
lions of  dollars.  If 
the  principles  outlined 
here  are  successfully 
applied  during  coming 
years  In  all  Air  Force 
development  programs, 
savings  In  dollars  and 
materials  will  be 
enormous. 


In  sunmary,  the  need  for  this  guidebook  is  derived  from  the  following 
observations  which  will  be  explained  and  substantiated  In  later  chap- 
ters : 


Growing  support  costs  have  motivated  top  level  managers  In 
the  Department  of  Defense  and  the  Air  Force  to  call  for  Im- 
proved RAM  of  all  systems  and  equipments,  and  to  support 
the  management  acts  needed  to  achieve  this  goal. 

Technology  Is  generally  available,  with  some  exceptions,  to 
achieve  the  goals  of  improved  RAM. 

Many  complex  equipments  have  achieved  high  RAM  goals,  but  many 
have  not.  In  the  deficient  programs  which  have  been  observed, 
the  program  decisions  which  led  to  RAM  problems  were  easy  to 
recognize,  relatively  few  in  number,  and  generally  managerial 
In  character. 

Program  and  project  managers  may  lack  .understanding  of  the 
elements  and  significance  of  a good  RAM  program. 

The  management  actions  needed  to  insure  an  adequate  RAM  pro- 
gram are  prescribed  In  current  regulations  and  standards,  are 
a good  Investment,  and  are  easy  to  understand  when  outlined 
and  explained  In  an  orderly  way. 
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SCOPE  and  ORGANIZATION 
of  the 
GUIDEBOOK 


SCOPE  AND  ORGANIZATION 


The  R&M  management  guidelines  which  follow  have  wide  application. 
Even  though  the  technical  approach  to  RAM  design  and  demonstration 
may  be  quite  different  in  various  technologies,  the  RAM  management 
issues  and  considerations  are  quite  similar.  Therefore,  whether 
you  are  working  on  electronic,  hydraulic,  or  mechanical  systems, 
the  management  guidelines  explained  should  be  largely  applicable 
to  your  program,  even  though  technical  details  will  be  different. 

Management  activities  and  issues  are  affected  by  the  development 
status  of  the  system  or  equipment,  and  consequently,  each  section 
of  the  report  will  consider  the  life  cycle  phases  from  concept  form- 
ulation to  deployment.  Anyone  unfamiliar  with  Air  Force  management 
of  acquisition  programs  and  the  phases  of  development,  should  first 
review  AFSC  Pamphlet  800-3,  A Guide  for  Program  Management,  avail- 
able from  the  Government  Printing  Office. 

While  this  guidebook  is  principally  for  AFSC  RAM  managers,  it  is 
written  with  the  fundamental  attitude  that  development  of  new  sys- 
tems and  equipments  is  a joint  venture  between  Air  Force  Systems 
Command  (AFSC),  Air  Force  Logistics  Command  (AFLC),  and  the  opera- 
tional or  using  command.  Of  course,  AFSC  is  in  the  driver's  seat 
until  transfer  of  program  management  responsibility  to  AFLC.  Trans- 
fer occurs  sometime  during  the  production  phase  at  a time  mutually 
agreeable  to  AFLC  and  AFSC.  Throughout  the  acquisition  phase  it  is 
important  that  the  new  system  is  developed  to  be  responsive  to  both 
operational  needs  and  long  range  logistic  support  needs.  At  the 
same  time,  the  new  system  is  built  from  a technology  base  which  is 
improving  continuously  through  research,  exploratory  and  advanced 
developments  sponsored  by  Government  agencies  and  private  industry. 
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This  is  a management  guidebook,  but  it  is  not  always  easy  to 
categorize  a particular  activity  as  strictly  ma.iagement  or 
strictly  technical.  Some  activities  will  fall  in  between. 

In  every  case,  the  management  significance  of  the  activity 
discussed  should  be  clear,  even  if  it  has  technical  facets. 

While  most  readers  will  have  considerable  experience  either 
managing  or  being  managed,  we  recommend  they  read  the  summary 
of  management  principles  in  Chapter  3.  The  chapters  which 
follow  it  are  written  using  the  terms  and  concepts  of  that 
summary. 

This  guidebook  is  written  to  aid  effective  implementation  of 
three  basic  directives.  They  are: 

Air  Force  Regulation  80-5  and  AFSC  Supplement  1, 

Air  Force  Reliability  and  Maintainability  Program 

MIL-STD-785, 

Reliability  Program  for  Systems  and  Equipment 
Development  and  Production 

MIL-STD-470, 

Maintainability  Program  Requirements 
(for  Systems  and  Equipments) 

There  are  many  other  regulations,  standards,  and  manuals  which 
make  reference  to  R&M  management  and  engineering,  but  the  above 
three  are  the  basic  ones  for  Air  Force  use,  and  they  in  turn 
reference  other  applicable  documents.  Throughout  this  book, 
other  useful  or  pertinent  documents  are  referenced  where  appro- 
priate. 

Chapter  2 explains  why  improvements  in  R&M  have  a high  payoff 
and  why  extraordinary  management  attention  is  needed  to  obtain 
that  payoff. 

Chapter  3 summarizes  traditional  and  time-proven  principles  of 
management  which  are  applied  in  subsequent  chapters  to  discus- 
sions of  R&M  management. 

Chapter  4 explains  the  elements  of  a standard  reliability  assur- 
ance program,  including  element  time  phasing,  relative  importance, 
relative  costs,  justification,  interdependence,  technical  nature, 
who  does  it  and  references  for  more  detailed  Information. 

Chapter  5 parallels  the  coverage  of  Chapter  4,  but  addresses  a 
standard  maintainability  assurance  program. 
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Chapter  6 summarizes  the  important  management  acts  needed  at 
headquarters  staff  levels,  the  system  program  office,  and  the 
contractor,  to  insure  successful  implementation  of  th*!  'l&M  pro- 
grams explained  in  Chapters  4 and  5. 

Chapter  7 discusses  the  relatively  new  area  of  computer  software 
R&M,  and  explains  guidelines  for  planning  an  orderly  program  for 
software  development  that  will  enhance  the  prospects  of  achieving 
R&M  goals. 

Appendix  A lists  sources  of  educational  opportunities  in  R&M. 
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Chapter  2 

MOTIVATION  FOR  THE  MANAGER 


INTRODUCTION 

This  chapter  shows 
why  the  R&M  manager 
must  be  motivated  to 
manage  his  functional 
activities  with  spe- 
cial care.  First  It 
shows  that  long  range 
cost  saving  benefits 
can  be  spectacular, 
and  that  R&M  engineer- 
ing knowhow  largely 
exists  to  achieve 
reasonable  objectives. 

However,  It  then  ex- 
plains that  human  na- 
ture and  program  pres- 
sures, In  both  the 
contractor  and  Air 
Force  organizations, 
tend  to  work  against 
achievement  of  those 

objectives.  Successful  programs  result  from  motivated  managers 
who  perceive  the  special  pitfalls  of  R&M  management  and  work 
around  them.  Later  chapters  show  how  they  do  It. 

R&M  ARE  MATURING  ENGINEERING  DISCIPLINES  ^ j 

Although  reliability  and  maintainability  considerations  under  var-  ^ 

lous  labels  have  existed  as  long  as  there  were  machines  to  cause  ; 

R&M  problems,  the  genesis  of  R&M  as  an  engineering  discipline  be-  j 

gan  In  World  War  II  when  the  complexity  of  electronic  equipment 
reached  the  point  where  R&M  became  a significant  concern  both  oper- 
ationally and  economically.  During  World  War  II,  most  reliability 
efforts  were  concerned  with  components,  primarily  vacuum  tubes. 

In  1950  a broader  attack  on  reliability  problems  was  Instituted  ! 

with  the  formation  of  the  Ad  Hoc  Group  on  Reliability  of  Electronics 
Equipment  under  the  Department  of  Defense  Research  and  Development 
Board,  succeeded  In  1952  by  the  Advisory  Group  on  Reliability  of 
Electronic  Equipment  (AGREE).  In  1956  the  first  reliability  design 
handbook,  "Reliability  Factors  for  Ground  Electronic  Equipment"  was 
published  by  the  Rome  Air  Development  Center  (RADC).  in  1957  AGREE 
published  their  report,  "Reliability  of  Military  Electronic  Equip- 
ment" which  provided  the  reliability  test  methods  still  In  use  to- 
day. In  1958  the  publication  by  RADC  of  a reliability  prediction 
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technique  based  on  electronic  parts  stress  data  completed  the 
set  of  basic  tooU  essential  to  reliability  engineering.  At 
this  point  a practical  reliability  engineering  discipline  ex- 
isted for  electronic  systems.  In  the  next  five  years  maintain- 
ability design,  prediction  and  demonstration  techniques  also 
became  available  and  the  R&M  engineering  discipline  was  firmly 
established. 

Efforts  subsequent  to  the  above  have  endeavored  to  continually 
improve  the  R&M  tools  in  a dynamic  environment  of  electronics 
development  (as  exemplified  by  the  explosive  development  of 
semiconductors)  and  have  reached  even  higher  degrees  of  sophis- 
tication in  attempts  to  reduce  the  cost  of  R&M  activities,  in- 
crease their  efficiency,  and  address  more  subtle  R&M  problems 
still  eluding  satisfactory  resolution.  These  efforts  have  in- 
cluded the  development  of  quality  assurance  procedures  for  elec- 
tronic semiconductor  devices,  prediction  techniques  for  equip- 
ment in  dormant  conditions  (i.e.,  a Minuteman  in  a silo),  tech- 
niques for  predicting  reliability  in  the  conceptual  stage  when 
only  gross  system  parameters  are  known,  more  efficient  statis- 
tical methods  to  reduce  test  time,  designs  for  built-in  fault 
indicators,  methods  of  system  design  to  accommodate  or  circum- 
vent failures,  and  of  course,  a significant  effort  merely  to 
keep  up  with  changes  in  parts,  manufacturing  methods,  and  im- 
proved parts  reliability.  Current  pioneering  efforts  involve 
development  of  R&M  growth  predictors,  quantification  of  the 
effects  of  field  environment  on  expected  reliability,  and  the 
R&M  of  computer  software.  Software  R&M  is  of  increasing  im- 
portance since  all  new  major  Air  Force  systems  are  software 
dependent  and  becoming  more  so  as  digital  equipments  are  put 
into  wider  use.  Development  of  tools  for  assuring  software 
R&M  may  be  the  next  major  step  forward  in  R&M  engineering. 
Finally,  human  reliability  and  human  factors  in  equipment  de- 
sign for  reliability,  while  hardly  a new  concern,  still  elude 
satisfactory  quantification. 

THE  PAYOFF  OF  RELIABILITY  & MAINTAINABILITY 


The  greatest  benefit  of  R&M  in  the  current  Air  Force  environ- 
ment is  its  economic  impact.  Of  course,  R&M  also  affect  safety 
and  the  probability  of  mission  success.  However,  both  of  these 
factors  can  be  forced  to  display  satisfactory  values  even  with 
equipments  of  relatively  poor  R&M  if  attendant  penalties  in  cost, 
weight,  power  consumption  and  support  effort  are  accepted.  For 
example,  redundant  subsystems,  larger  force  sizes,  greater  quan- 
tities of  spares  and  expanded  maintenance  shops,  can  overcome 
poor  R&M  as  far  as  mission  success  and  safety  are  concerned. 
However,  the  cost  penalties  of  these  approaches  while  always 


undesirable  are  now  intolerable.  The  following  paragraphs 
describe  the  critical  economic  situation  faced  by  the  Air 
Force  and  the  ways  in  which  improved  R&M  can  relieve  that 
situation. 

THE  BUDGET  CRUNCH 

The  economic  realities  facing  the  Air  Force  are  illustrated 
in  the  following  chart  which  shows  the  Department  of  Defense 
(DoD)  budget  for  Baseline  Forces  (i.e.,  with  additional  costs 
of  Vietnam  operations  removed)  in  terms  of  current  dollars 
and  constant  1979  dollars  for  the  last  10  fiscal  years. 


As  the  chart  illustrates,  while  the  Defense  budget  was 
doubled  in  10  years,  the  actual  purchasing  power  in  FY-79  is 
less  than  that  of  FY-69.  The  overall  trend  has  been  a constant 
decline  in  purchasing  power  except  for  the  last  two  years  when 
the  trend  reverses.  This  reflects  the  concern  over  a steady 
increase  in  Defense  spending  by  the  competition,  which  is  also 
reflected  in  a request  by  President  Carter  that  all  NATO  nations 
increase  their  budgets  by  3%  a year  over  inflation.  If  pursued, 
this  policy  should  result  in  real  increase  in  the  Defense  budgets 
of  the  future.  However,  the  impact  of  the  decline  will  be  felt 
for  some  time.  In  1968  the  Air  Force  had  12,606  aircraft.  In 
1978  we  had  7,290.  As  a result,  mission  success  cannot  be 
assured  simply  by  assigning  more  aircraft  to  a job.  Instead 
"force  multipliers"  are  sought.  These  are  means  for  improving 
the  capability  of  a limited  force  without  increasing  its  size. 

The  application  of  R&M  techniques,  leading  to  increased  sortie 
generation  rates,  is  one  of  these  multipliers. 

Returning  to  cost  considerations,  another  problem,  other 
than  the  sheer  number  of  dollars  available,  is  the  fact  that 
the  portion  of  the  budget  required  for  operating  and  maintenance 
costs  is  increasing  at  the  expense  of  procurement  costs.  The 
following  charts  show  some  interesting,  if  disturbing  data. 


AIR  FORCE  BUDGET  AND  ASSETS 
BUDGET  (MILLIONS  OF  DOLLARS) 

IN  CURRENT  YEAR  DOLLARS  IN  FY-79  DOLLARS 


FY-68 

FY-69 

CHANGE 

FY-68 

FY-78 

CHANGE 

Total  Budget 

24,947 

33,200 

+32.9% 

52,931 

35,145 

-33.6% 

Procurement 

9,071 

10,407 

+14.7% 

19,221 

11,017 

-42.7% 

0AM 

6,170 

10,037 

+62.7% 

13,027 

10,625 

-18.8% 

ASSETS  (PERSONNEL  IN  THOUSANDS) 


FY-68 

FY-78 

CHANGE 

Aircraft 

12,606 

7,290 

-42.2% 

Military  Personnel 

905 

571 

-36.9% 

Civilian  Personnel 

342 

253 

-26.0% 

Total  Personnel 

1,247 

824 

-33.9% 

14 


Comparing  FY-68  to  FY-78  in  current  year'  dollars  shows 
the  O&M  costs  increasing  four  times  as  fast  as  procurement  costs. 
In  FY-68  procurement  dollars  were  about  50%  higher  than  04M, 
while  in  FY-78  they  are  roughly  equal.  In  actual  spending  power, 
obtained  by  converting  current  year  dollars  to  equivalent  FY-79 
dollars,  the  33.6%  decrease  in  the  Air  Force  budget  is  exceeded 
by  the  decrease  in  procurement  funds  while  O&M  funds  show  a 
lesser  drop.  Put  another  way,  in  FY-78  procurement  used  36.3% 
of  the  Air  Force  budget  while  O&M  costs  took  24.7%.  Today  both 
take  about  30%  of  the  budget.  Hence,  we  are  spending  more  on 
our  current  equipment  and  less  on  developing  new  systems.  In 
an  age  of  dramatically  expanding  military  technology,  this  is 
a bad  scene. 


The  solution  is  even  worse  when  the  lower  half  of  the  chart 
is  considered.  One  would  expect  a shift  of  dollars  from 
procurement  to  O&M  if  the  budget  declines  and  the  capability 
remained  constant.  However,  the  aircraft  and  personnel  in 
FY-79  were  more  reduced  than  the  budget.  Hence,  the  O&M  costs 
are  increasing  faster  than  inflation,  and  if  we  had  a budget 
adjusted  for  inflation  to  provide  a constant  purchasing  power, 
we  would  still  find  O&M  costs  pulling  dollars  away  from  procurement 
as  time  passed.  This  trend  must  be  checked  and  better  R&M  is 
one  of  the  means  for  doing  so. 

Another  factor  compounding  the  problem  is  that  the 
procurement  dollar  also  buys  less  than  it  used  to,  even  account- 
ing for  inflation.  In  1959  the  B-52  bomber  cost  about  $10 
million.  The  B-1  was  expected  to  cost  around  $60  million.  Half 
of  the  difference  can  be  attributed  to  inflation.  The  other 
half  was  caused  by  greatly  increased  capability.  This  capability 
is  provided  by  complex,  sophisticated  systems  which  are  costly 
to  design,  produce  and  maintain. 


By  this  time,  one  should  be  convinced  that  reduced  costs 
are  an  urgent  Air  Force  need.  In  the  next  paragraphs  we 
will  look  at  the  role  of  R&M  in  cost  reduction. 


REDUCING  COSTS  THROUGH  R&M 

The  impact  of  Reliability  on 
O&M  costs  is  obvious.  The 
more  failures,  the  more 
resources  needed  to  maintain 
the  equipment.  But  since 
there  is  a cost  to  a 
reliability  program,  it 
must  be  looked  at  as  an 
investment,  and  shown  to 
be  worthwhile. 
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BENEFIT 
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An  analysis  of  reliability  as  a capital  investment  was 
published  in  a technical  paper  at  the  1974  Annual  Reliability 
and  Maintainability  Symposium  (reference  1).  The  author  of 
that  paper  studied  in  detail  the  development  and  performance 
history  of  three  production  equipments.  These  equipments 
were  non-digital  avionic  subsystems  used  in  high-performance 
aircraft,  and  each  incorporated  from  3,300  to  13,500  parts 
and  each  cost  from  $35,000  to  $243,000  in  production.  He 
made  a detailed  cost  analysis  of  a thorough  reliability  program 
that  could  have  been  followed  in  development  and  production, 
and  then  estimated  the  resulting  reliability  improvements.  These 
reliability  improvements  then  yielded  projected  reductions  in 
maintenance  costs,  and  reductions  in  acquisition  costs  because 
fewer  spare  parts  were  needed.  Conservative  assumptions  were 
used  throughout  that  tended  to  increase  the  predicted  costs  and 
diminish  the  predicted  benefits.  The  following  table  sunmarizes 
the  results: 


RELIABILITY  AS  A CAPITAL  INVESTMENT 


CASE 

QUANTITY 

ADDITIONAL 
RELIABILITY 
PROGRAM  COST 

SPARE  PARTS 
SAVINGS 

INITIAL 

NET 

COST 

ANNUAL 
MAI NT. 
SAVINGS 

10- YEAR 

TOTAL 

SAVINGS 

I 

564 

$9.5M 

$4.5M 

$5.0M 

$5.0M 

$44. 7M 

11 

325 

$2.1M 

$ .5M 

$1.6M 

$ .7M 

$ 5.4M 

III 

335 

$6.9M 

$4.1M 

$2.8M 

$2.5M 

$22. 2M 

From  < 

» businessman's  point  of  view,  the  yearly  returns  on  the 

investment  range  from  44%  to  100%--returns  nobody  would  ever  pass 
up. 


Maintainability,  the  other  half  of  R4M,  obviously  impacts  the 
cost  of  maintenance  resources.  While  reliability  dictates  how 
often  maintenance  will  be  performed,  maintainability  dictates 
how  much  it  will  cost.  Maintainability  considerations  include 
not  only  reducing  maintenance  time  (by,  for  example,  assuring 
access  to  failed  parts),  but  designing  to  reduce  associated  costs 
such  as  test  equipment.  A recent  RADC  study  of  digital  printed 
circuit  board  test  requirements  (reference  2)  showed  that  almost 
all  Air  Force  digital  board  test  needs  could  be  met  by  available 
testers.  Yet,  special  purpose  testers  are  too  frequently  procured, 
at  exorbitant  prices.  A $4  million  special  digital  printed  circuit 
board  tester  was  proposed  for  the  427-M  system,  but  due  consideration 
of  the  cost  revealed  a $1  million  tester,  commercially  available, 
was  adequate.  The  TRI-TAC  system  deliberated  between  an  $800,000 
tester  or  a $15,000  tester  and  found  the  $15,000  tester  will  do 
the  job.  While  the  savings  in  this  case  may  be  more  of  an  accident 
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than  a deliberate  event.  It  does  show  that  design  to  use 
low  cost  available  testers  can  bring  significant  savings. 

The  pay-off  of  R&M  is  also  evidenced  by  the  Air  Force  Producibility, 
Reliability,  Availability  and  Maintainability  (PRAM)  Project  Office. 

This  was  created  in  August  1975,  as  a combined  AFLC  and  AFSC 
organization.  Its  function  is  to  invest  in  improvements  of  Air 
Force  in-service  weapons  systems  to  reduce  ownership  costs.  As 
of  31  Aug  78,  PRAM  has  invested  $42  million  in  379  projects  with 
a projected  five  year  net  savings  of  over  $795  million.  Quite  an 
impressive  return. 

THE  CURRENT  R&M  REPORT  CARD 


Despite  the  impressive  benefits  of  R&M  discussed  above,  present 
programs  are  not  taking  advantage  of  these,  according  to  a survey 
taken  by  AFALD.  The  study  was  completed  Nov  78  and  presented  at 
the  AFSC  Reliability  and  Information  Exchange  Meeting,  28-30  Nov  78. 
The  survey  found  that  of  162  programs,  109  had  satisfactory 
reliability  provisions,  33  were  marginal  and  20  unsatisfactory. 

The  major  problems  identified  were  contract  reliability  requirements, 
reliability  demonstrations  and  parts  control.  Evidently,  there  is 
room  for  improvement  in  reliability. 

The  same  conclusion  could  be  reached  from  a GAO  letter  report, 
"OoD  Standardization  of  Avionics  and  Other  Electronics"  (OSD 
Case  #4732,  PSAD  78-105)  which  listed  in  its  findings  "low 
reliability  of  avionics  is  often  a factor  in  the  readiness  of 
operational  weapon  systems  and  could  hinder  effective  military 
operation". 

On  maintainability,  the  current  trend  to  built-in-test  (BIT) 
as  a means  for  rapidly  detecting  and  evaluating  failures  has 
produced  its  own  problems.  In  1977,  Maj  Gen  Howard  W.  Leaf, 

Commander  of  AFTEC,  expressed  concern  over  the  BIT  system  for 
the  Wild  Weasle  and  the  Central  Integrated  Test  Subsystem  (CITS) 
for  the  B-1.  Lt  Gen  Robert  T.  Marsh,  Commander  of  ESD,  cited  a 
need  for  a phased  approach  with  measurable  milestones  for  Built- 
in-Test  development  and  demonstration.  Lt  Gen  Robert  C.  Mathis, 

Vice  Conmander  of  AFSC,  stated  the  belief  that  real  improvements 
in  fault  detection/isolation  are  achievable,  but  that  it  is  a 
major  cost/management  challenge. 

Some  of  these  problems  are  of  course  technical  (e.g.,  an 
urgent  need  exists  for  development  of  techniques  for  demonstrating 
BIT  capability).  Some,  however,  are  managerial.  The  concern 
expressed  above  resulted  in  an  ASD  program  to  survey  and  assess 
the  BIT  capability  of  existing  systems,  an  RADC  program  to 
develop  methods  and  tools  for  designing  cost-effective  fault 
detection  and  isolation  subsystems,  and  a set  of  interim  guidelines 


for  program  managers  prepared  for  ESD  by  RADC.  Clearly, 
maintainability  engineering  still  requires  technical  and  manage- 
ment attention. 

OTHER  R&M  BENEFITS 

Mention  has  been  made  of  the  impact  of  R&M  as  operational  readiness. 
The  next  four  paragraphs  will  expand  this  to  the  concept  of 
operational  effectiveness  and  will  discuss  some  other  benefits  of 
attention  to  R&M. 

R&M  IMPACT  ON  OPERATIONAL  EFFECTIVENESS 

Every  operational  commander  wants  reliable  equipment,  and  if 
something  does  go  wrong,  he  wants  it  fixed  quickly.  In  other 
words,  he  knows  that  his  operational  effectiveness  depends  upon 
R&M.  He  wants  his  systems  to  be  avai lable,  dependable,  and 
capable  for  the  mission. 

The  operational  effectiveness  of  a system  may  be  defined  to  be 
a function  of  availability  (the  probability  that  the  system  will 
be  in  an  operating  state  at  the  start  of  a mission),  dependability 
(the  probability  that  it  will  remain  in  a satisfactory  operating 
state  for  the  length  of  the  mission)  and  capability  (the 
probability  that,  if  in  a satisfactory  operating  state,  it  will 
successfully  perform  the  mission).  Dependability  is  obviously 
derived  from  reliability.  Both  reliability  and  maintainability 
impact  availability.  These  interrelationships  are  illustrated 
below: 


DESIGN 

^ PHYSICAL 

RELIABILITY > MAINTAINABILITY  PERFORMANCE 
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The  dashed  line  in  the  diagram  illustrates  that  maintainability 
is  analytically  a function  of  reliability.  That's  because 
subassembly  reliability  affects  the  computed  maintainability 
parameter,  mean-time-to-repair  (MTTR),  of  an  equipment.  To 
compute  the  MTTR  of  an  equipment  made  up  of  many  repairable 
subassemblies,  the  MTTR  of  each  subassembly  is  weighted  by  a co- 
efficient which  depends  on  the  mean-time-between- failure  (MTBF) 
of  each  subassembly.  During  equipment  design  for  maintainability, 
this  dependence  of  equipment  MTTR  on  subassembly  MTBF  is  used 
along  with  logic  constraints  to  partition  the  equipment  into 
repairable  subassemblies  in  a way  that  will  give  the  best  MTTR 
for  the  integrated  equipment.  Of  course,  equipment  reliability 
is  in  turn  affected  by  the  maintenance  concept  or  design  because 
of  human  factors— the  way  people  handle  things. 

In  summary,  it  is  clear  that  reliability  and  maintainability 
affect  operational  effectiveness,  even  though  the  more  obvious 
effects  of  physical  or  dynamic  performance  tend  to  get  much  more 
attention.  Furthermore,  reliability,  maintainability  and  physical 
performance  are  attributes  which  can  be  designed  into  a system  or 
equipment,  and  all  depend  on  the  environment  in  which  the  equipment 
operates  or  is  repaired.  In  short,  poor  R&M  can  cancel  the 
superior  operational  effectiveness  you  had  hoped  to  achieve 
through  better  physical  performance. 

LOGISTIC  SUPPORT  PLANNING  NEEDS  DEPENDABLE  R&M  DATA 

Integrated  Logistics  Support  (ILS)  plans  depend  heavily  on 
predictions  of  R&M  field  performance.  Obviously,  plans  for 
such  things  as  spare  parts,  maintenance  facilities,  field  and 
depot  maintenance  equipment,  and  maintenance  personnel  depend 
directly  on  predicted  MTBF  and  MTTR  of  the  equipments  and  sub- 
assemblies,  Without  a complete  R&M  program,  prediction  of  R&M 
field  performance  is  largely  guesswork  or  wishful  thinking.  That's 
because  development  and  production  contract  R&M  requirements  will 
have  no  predictable  relationship  to  field  results  if  equipment  R&M 
design  is  not  systematically  tailored  to  those  requirements  during 
development,  and  if  R&M  achievement  is  not  continuously  measured 
in  development  and  production.  This  situation  has  been  illustrated 
many  times  in  the  history  of  electronic  equipment  development  with 
cases  where  field  reliability  differed  from  predictions  used  for 
logistics  planning  by  factors  as  high  as  ten  to  twenty,  usually 
for  the  worse.  This,  of  course,  leads  to  major  disruption  of 
support  at  the  beginning  of  deployment,  and  subsequent  delays 
and  financial  crises  before  reaching  the  system  effectiveness 
originally  predicted.  On  the  other  hand,  a solid  R&M  program  will 
pay  off  with  reduced  ILS  planning  uncertainty,  and  prevention  of 
such  disasterous  surprises  in  the  field.  Overly  optimistic  R&M 
statements  result  in  not  operationally  ready  due  to  lack  of 
spares  (NORS)  conditions,  but  of  equal  importance,  pessimistic 
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statements  of  R&M  result  in  the  over-buy  of  spares  which  the 
Air  Force  can  ill  afford. 

THE  FRINGE  BENEFIT  DURING  DEVELOPMENT  TESTS 

Good  R&M  clearly  benefits  both  the  Air  Force  and  the  contractor. 

That  is,  good  R&M  performance  during  development  testing  will 
insure  that  tests  move  along  smoothly  and  that  engineers  can 
concentrate  on  verifying  physical  performance  parameters.  Good 
reliability  techniques  (such  as  incorporating  high  reliability 
parts)  should  not  be  delayed  until  the  qualification  test  articles. 
The  more  reliable  the  early  prototypes  are,  the  more  effective  and 
efficient  the  test  program  and  the  earlier  the  equipment  reliability 
can  be  evaluated.  Nothing  is  more  aggravating  during  a test 
program  than  unexpected  equipment  failures  which  have  nothing  to 
do  with  the  purpose  of  the  test.  Such  failures  lead  to  expensive 
program  delays,  or  frustrating  decisions  on  whether  or  not  to 
cut  short  some  tests  to  avoid  these  delays.  In  this  situation, 
both  the  Air  Force  program  office  budget  and  the  contractor's  profit 
can  be  jeopardized  by  poor  R&M.  Furthermore,  the  reputation  of  both 
the  Air  Force  program  manager  and  the  contractor  are  usually  at 
stake  during  this  time  and  a production  decision  may  be  in  the 
balance.  A well -planned  and  executed  R&M  program  can  pay  handsome 
benefits  in  time  savings,  cost  avoidance,  profit  and  reputations 
during  this  critical  period. 

WHY  R&M  NEEDS  SPECIAL  MANAGEMENT  EMPHASIS 


With  all  the  benefits  discussed,  one  would  think  that  R&M  would 
be  a high  priority  of  all  program  managers.  Unfortunately,  this 
is  not  so.  Some  of  the  pressures  acting  against  attention  to  R&M 
are  discussed  below: 

PERFORMANCE  IS  EXCITING 

First  of  all,  America  has  tended  to  be  a throw-away 
society  with  little  concern  for  conservative  use  of  natural 
resources.  We  buy  a flashy  new  car  or  appliance,  use  it  for 
several  years,  and  when  it's  worn  out  we  trade  it  in  for  a small 
faction  of  its  cost  or  throw  it  away.  We  have  not  been  a 
society  where  durability  stood  high  on  our  scale  of  values, 
although  there  is  some  recent  evidence  that  maybe  this 
attitude  is  changing.  On  the  other  hand,  dynamic  and  physical 
performance  is  put  high  on  our  scale  of  values--speed,  size, 
acceleration,  agility,  flexibility,  range,  style,  fidelity, 
and  so  on.  This  sense  of  values,  developed  from  childhood  on, 
probably  has  a subtle  affect  on  attitudes  toward  durability 
efforts  in  Air  Force  development  programs.  Anyone  who  has 
worked  in  one  of  these  programs  will  surely  agree  that  management 


interest  is  most  intense  when  discussing  the  system's  acceleration, 
speed,  range,  altitude,  fire-power,  flexibility,  load  capacity, 
sensitivity,  signal-to-noise  ratio,  and  so  forth,  and  its  resulting 
effectiveness  in  a warfare  engagement  analysis.  Reliability  and 
maintainability  on  the  other  hand  are  often  treated  as  just  a 
couple  of  the  "ilities"  that  the  contract  "boiler  plate"  will 
take  care  of.  This  atti tude'has  been  reversed,  however,  in 
certain  programs  where  reliability  is  a life  or  death  matter  for 
the  program--certain  space  and  missile  programs  for  example,  and 
equipments  where  flight  safety  is  a paramount  consideration.  In 
any  case,  unusual  motivation  is  needed  to  elevate  R&M  to  the  level 
of  attention  that  long  range  economics  show  they  deserve. 

R&M  BENEFITS  ARE  IN  THE  FUTURE 

Another  factor  which  tends  to  diminish  interest  in  R&M 
activities  is  the  long-term  nature  of  most  of  the  payoffs.  The 
benefits  of  these  activities  are  not  immediately  visible  during 
development  and  production.  An  R&M  program  costs  money,  but  the 
big  payoff  doesn't  come  until  years  later  in  the  form  of  a reduced 
logistics  budget.  The  program  manager  on  the  other  hand  may  be  over- 
whelmed by  the  many  short-term  goals  he  must  strive  to  achieve. 
Examples  are:  meeting  next  year's  budget,  meeting  initial  delivery 
and  flight  dates,  and  passing  physical  performance  tests.  Also, 
the  program  manager's  performance  may  be  judged  on  the  basis  of 
achievement  of  these  easily  quantified  short-term  goals.  Conse- 
quently, the  program  manager  must  be  motivated  to  insure  that  R&M 
gets  the  management  emphasis  that  long-range  Air  Force  interest 
demand,  since  the  development  and  production  steps  may  span  four 
to  eight  years  and  the  total  life  cycle  may  span  20  years  or  more. 
Long-term  cost  saving  goals  of  the  Air  Force  must  be  translated 
into  short-term  management  goals  which  will  successfully  attract 
the  manager's  attention. 

INITIAL  R&M  ACTIVITY  IS  LOW-KEY 

R&M  engineering  is  not  a very  flashy  business,  especially  dur- 
ing initial  design  and  fabrication.  That  is,  the  work  is  detailed 
and  complex,  and  milestones  are  not  very  large.  Progress  or  lack 
of  it  in  the  early  and  very  important  phases  is  not  easy  to  measure. 
Consequently,  management  may  give  it  only  casual  notice.  An  inter- 
mediate period  activity  which  may  arouse  some  management  interest 
is  nonstandard  parts  approval  requests  from  the  contractor.  Here 
the  Air  Force  program  manager  must  make  some  decisions  which  have 
major  long-term  ramifications,  but  he  may  be  tempted  to  capitulate 
to  those  ever  present  short-term  pressures  such  as  budget  and 


schedules.  It  Is  not  until  R&M  demonstration  tests  that  tangible 
and  Iwnedlate  problems  are  apparent--fa11ure  to  pass  the  tests. 

By  then,  the  program  has  momentum,  hardware  Is  being  produced  In 
quantity,  the  contractor  Is  Itching  for  a follow-on  procurement, 
any  any  delays  or  significant  equipment  modifications  mean  impacts 
on  schedule  and  budget--the  pressure  Is  on  for  a compromise  of  RAM 
requirements.  So  ^e  can  see  again,  there  Is  need  for  special  In- 
centives and  motivation  to  give  RAM  the  management  attention  they 
deserve  from  the  very  beginning  of  the  program. 

NO  EASY  ANSWERS 

Can  the  program  manager  rely  on  the  contractor  to  meet  RAM  re- 
quirements? Not  entirely.  The  contractor  Is  motivated  by  profit 
and  survival  of  the  company,  and  these  usually  translate  Into  short- 
term objectives  so  far  as  a particular  project  manager  Is  concerned. 
Some  companies  have  good  RAM  engineering  staffs,  but  you  will  get  no 
more  effort  than  you  specify  for  delivery  under  the  contract,  and 
any  design  or  test  alternatives  not  explicitly  stated  In  the  contract 
can  be  expected  to  be  selected  by  the  company  staff  to  benefit  com- 
pany Interests.  The  Air  Force  must  look  out  for  Its  own  long-term 
Interests.  Contract  provisions  such  as  warranties  and  performance 
Incentives  can  provide  limited  Insurance  that  Air  Force  long-term 
Interests  art  protected,  but  as  discussed  In  Chapter  6,  these  pro- 
visions must-be  supplemented  by  other  RAM  engineering  tasks,  both 
contractor  and  Government,  tailored  to  the  particular  equipment. 

Can  the  program  manager  rely  on  the  long  list  of  standards,  spec- 
ifications, and  manuals  In  the  contract?  Again,  not  entirely.  The 
military  standards  and  specifications  for  RAM  contain  numerous  al- 
ternate procedures  and  requirements  so  that  they  will  be  adaptable 
to  all  kinds  of  equipment  situations.  The  program  manager  must  pro- 
vide for,  and  must  strongly  support,  an  RAM  engineering  advisory 
team  which  can  intelligently  select  the  critical  RAM  requirements 
to  put  In  the  contract,  and  then  follow  through  with  surveillance 
of  contractor  performance.  This  team  must  also  help  the  program 
manager  make  program  decisions,  such  as  choice  of  design  and  test 
alternatives,  which  he  may  not  fully  understand  from  a technical 
standpoint.  Standards,  specifications,  and  manuals  provide  the 
fabric  from  which  to  tailor  an  RAM  program  suited  to  the  particular 
equipment,  and  this  tailoring  must  be  done  by  a team  of  specialists 
motivated  to  protect  the  long-term  interests  of  the  Air  Force. 

From  all  of  the  above,  it  is  clear  that  we  must  have  motivated  Air 
Force  program  managers  who  perceive  the  special  pitfalls  of  RAM 
management  and  then  know  how  to  work  around  them.  Those  pitfalls 
include  the  American  passion  for  performance  which  distracts  atten- 
tion,the  "ility"  image  of  RAM  which  implies  that  the  staff  will  take 
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care  of  it  somehow,  the  long-term  payoff  of  good  R&M  which  tends 
CO  diminish  short-term  interest,  the  low-key  analytical  character 
of  initial  R&M  work  which  makes  it  "dull",  and  the  temptation  to 
grasp  for  easy  answers. 

Consequently,  R&M  require  extraordinary  management  emphasis  and 
the  chapters  which  follow  explain  what  to  do  to  insure  that  we 
cash  in  on  the  spectacular  cost  saving  benefits  of  improved  R&M, 

SYNOPSIS 


The  shrinking  buying  power  of  the  Air  Force  budget  demands 
a variety  of  management  actions  to  reduce  costs. 

Improved  R&M  will  enhance  the  value  of  the  budget  b>  reatly 
reducing  long-term  operating  and  maintenance  costs. 

Case  histories  and  studies  show  clearly  that  the  cost  of  a 
good  R&M  program  is  an  excellent  long-term  investment  for  the 
Air  Force.  The  payoff  is  spectacular. 

Operational  effectiveness  of  systems  is  improved  by  better 
R&M  performance. 

The  short-term  pressures  on  a program  manager,  such  as  budget 
and  schedule,  often  conflict  with  the  Air  Force's  interests  in 
life-cycle  savings  through  good  R&M  performance. 

R&M  often  suffer  from  the  "ility"  or  "boiler  plate"  image. 

They  need  center- stage  attention  from  the  beginning  of  the  program. 

So  far,  no  simple,  concise  contractual  provision  has  been 
found  to  guarantee  long-term  R&M  performance.  The  program  manager 
needs  expert  advice  to  guide  the  way  and  monitor  the  progress. 

R&M  need  extraordinary  management  motivation  at  staff  levels, 
as  well  as  at  the  program  office,  in  order  to  secure  the  handsome 
long-term  money-saving  benefits  which  the  Air  Force  must  have, 
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Chapter  3 

MANAGEMENT  FUNDAMENTALS 


INTRODUCTION 


This  chapter 
gives  a brief 
explanation 
of  the  prin- 
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Whether  you 
manage  an 
Air  Force 

project  office,  a hardware  store  or  a football  team,  you  will 
find  that  all  of  your  management  activities  can  be  classified 
into  the  categories  explained  here.  If  you  find  that  one  of 
the  categories  is  not  covered  by  any  of  your  activities,  you 
may  be  headed  for  trouble  because  a major  management  function 
is  being  overlooked. 


Even  though  essentially  all  Air  Force  and  industrial  managers 
have  some  formal  education  in  management,  and  often  consider- 
able experience,  a quick  read-through  of  this  chapter  will  be 
helpful  because  it  explains  terminology  and  concepts  we  have 
elected  to  use  throughout  this  guidebook.  Various  authors 
discuss  management  somewhat  differently,  but  in  most  cases  it 
is  only  a relatively  minor  variation  on  the  theme  used  here. 

Management  can  be  defined  as: 

The  process  of  motivating  and  coordinating  an 
appropriate  group  of  people  to  perfonr.  ..he 
actions  necessary  to  achieve  a desired  set  of 
objectives. 

In  this  definition,  the  word  "coordinating"  needs  to  be  given 
special  meaning  and  emphasis.  Coordinate  means  here  that  the 
necessary  acts  must  at  least  supplement  one  another  (add  to), 
and  preferably  compliment  one  another  (amplify  or  multiply  each 
other)  so  as  to  yield  a synergistic  effect.  That  is,  the  most 
productive  organization  will  tend  to  have  many  complimentary 


activities  with  the  remainder  supplementary  and  none  counter- 
productive (subtractive).  The  organization  of  people  should 
produce  results  greater  than  the  sum  of  the  contributions  of 
its  individual  members. 

We  can  take  the  above  definition  of  management,  and  from  it 
list  five  categories  of  activity  which  collectively  account 
for  all  the  functions  of  management  expressed  or  implied  in 


the  definition.  These  categori 
Activity  Category 
Planning 

Organizing 

Manning 


and  their  contributions  are: 

Contribution  to  "Management" 

Establishes  the  organization 
objectives,  and  the  approach, 
policies,  rules  and  resources 
for  achieving  them 

Defines  duties,  responsibili- 
ties, authority,  and  the  co- 
ordinating relationships  be- 
tween people. 

Secures  an  appropriate  group 
of  people  able  to  perform 
needed  duties 


Leading  Instructs,  directs,  coordinates, 

and  motivates  people  to  perform 
needed  duties  to  progress  towards 
established  objectives  using  set 
resources,  policies  and  rules. 

Controlling  Measures  progress  towards  objec- 

tives and  takes  corrective  action 
to  remove  unwanted  deviations 

You  may  wonder  what  happened  to  "communicating,"  sometimes  listed 
as  a class  of  management  activity  by  some  authors.  We  consider 
communicating  as  fundamental  to  all  human  endeavor  and  obviously 
must  be  carried  out  to  execute  all  of  the  five  management  func- 
tions above.  Of  course,  communicating  is  also  necessary  in  engi- 
neering, law,  medicine,  football,  and  plumbing.  In  management, 
communicating  becomes  most  crucial  in  the  "leading"  function  as 
explained  later,  but  is  an  important  element  of  all  management 
activities.  There  is  no  doubt  that  a good  manager  is  also  a good 
communicator  who  is  skilled  in  employing  all  the  graphic  and  ver- 
bal forms  of  communications. 
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It  is  important  for  the  manager  to  view  communication  as  a 
two-way  process  which  includes  a message  and  a response  to 
that  message. 


1 


f 

i; 
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Simply  sending  out  a message  is  no  guarantee  that  communication 
has  occurred.  For  example,  the  daily  bulletin  is  not  a communi- 
cation, but  just  a message.  It  becomes  a communication  medium 
when  its  originators  get  some  kind  of  response  to  its  content. 

When  talking  on  the  telephone,  you  are  never  sure  the  other 
party  hears  or  understands  you  until  you  hear  an  acknowledge- 
ment that  reflects  comprehension  of  your  message.  Similarly, 
an  Air  Force  regulation,  specification,  or  directive  is  not  a 
communication,  but  just  a message.  There  is  no  assurance  of 
communication  through  these  documents,  unless  response  is  ob- 
served or  measured  in  some  way,  and  then  the  communication  ef- 
fectiveness might  turn  out  to  be  very  good  or  very  bad.  When 
we  send  out  a management  message  for  the  purpose  of  communica- 
tion, we  should  be  sure  to  provide  for  a measure  or  observation 
of  its  response  in  some  form. 

Getting  back  to  terminology,  textbooks  often  list  our  "manning" 
function  as  "staffing,"  but  "staffing"  has  special  meaning  in 
Air  Force  jargon  and  therefore  is  not  used  here.  The  texts  also 
commonly  use  the  term  "directing"  rather  than  our  "leading,"  but 
that  is  mostly  a matter  of  preference.  "Leading"  is  preferred 
here,  because  it  implies  "leadership"  and  motivation  as  well  as 
giving  directions.  The  remaining  terms  used  here  for  the  manage- 
ment functions— planning,  organizing,  and  control ling--are  widely 
used. 

The  five  management  functions  are  explained  briefly  in  the  follow- 
ing paragraphs,  and  some  comments  about  program  management  and 
systems  engineering  conclude  the  chapter. 
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PLANNING 


Planning  is  done  by  essentially 
everyone  in  an  organization,  but 
the  process  begins  at  the  top. 

In  fact,  it  will  begin  before  a 
formal  organization  exists. 

Planning  consists  generally  of 
examining  alternatives  along 
with  their  respective  risks, 
and  choosing  the  best  ones. 

This  assessment  and  selection 
of  alternatives  covers  objec- 
tives, policy,  strategy,  or- 
ganization, resources,  proced- 
ures, manning,  incentives,  and 
management  control  methods.  In 
other  words,  planning  encompasses 
and  is  involved  in  all  of  the  man- 
agement functions. 

The  first  and  most  important  al- 
ternatives to  consider  in  top 
level  plans  are  the  organization 
objectives.  Without  these,  no 
one  knows  where  they  are  headed  or  what  they  are  supposed  to 
accomplish.  The  objectives  should  be  clear,  concise,  specific 
and  expressed  in  quantifiable  or  measurable  terms,  so  that  sub- 
ordinates can  easily  understand  them  and  progress  towards  them 
can  be  measured  readily.  In  addition,  the  planning  premises, 
assumptions,  or  ground  rules  on  which  the  objectives  are  based 
should  be  literally  attached  to  the  objectives  to  aid  re-eval- 
uation of  the  objectives  should  the  premises  later  change. 
Finally,  the  objectives  must  be  prioritized  in  some  way.  Pri- 
orities serve  as  a guideline  (along  with  risk  assessment)  for 
the  allocation  of  resources,  and  also  guide  the  preparation  of 
derivative  plans  by  subordinates. 

The  objectives  must  be  explained  and  publicized  throughout  the 
organization  so  that  derivative  planning  objectives  can  be  de- 
veloped by  subordinates.  (Publicizing  plans  is  an  activity 
under  the  "leading"  category  discussed  below).  Once  objectives 
have  been  set,  everyone  should  know  "what  to  strive  for." 

Additional  planning  is  needed  to  establish  the  overall  strategy, 
policies,  procedures  and  resources  for  accomplishing  the  objec- 
tives. That  is,  "how  shall  we  do  it"?  Answering  this  question 
makes  it  obvious  that  planning  must  consider  the  other  manage- 
ment functions  of  organizing,  manning,  leading,  and  controlling. 
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Planning  actually  sets  the  stage  for  all  of  these  functions. 
Planning  will  yield  the  overall  organization  policies,  pro- 
cedures, ground  rules  and  resources  for  accomplishing  those 
functions  in  a way  which  contributes  to  the  objectives,  and 
more  detailed  derivative  plans  will  be  developed  down  through 
the  organization  to  carry  on  the  day-to-day  management  func- 
tions. 

In  general,  planning  activities  can  be  grouped  into  two  major 
categories,  "strategic  plans"  and  "operating  plans."  The 
kinds  of  questions  to  ask  and  typical  end  results  are  as  fol- 
lows: (there  is  not  necessarily  a one-to-one  relationship 
between  questions  and  results). 


Strategic  Plans 


Questions 


Results 


What  are  the  assumptions? 
What  are  the  risks? 

What  shall  we  do? 

How  important  is  it? 

How  should  we  approach  it? 
What  are  the  ground  rules? 
What  do  we  need? 


Premises 

Objectives 

Priorities 

Strategy 

Policies 

Resource  needs 

Milestones 


Operating  Plans 


Questions 


Results 


What  activities  are  needed? 
How  do  we  coordinate? 

What  are  the  tasks? 

Where  do  we  find  the  people? 
When  does  it  need  to  be  done? 
What  are  the  key  events? 

How  do  we  gauge  progress? 

How  much  money? 

What  facilities? 


Functions 
Organization 
Job  statements 
Recruiting  plan 
Schedules 
Procedures 
Control  methods 
Resource  allocations 
Budgets 


Plans  will  be  made  by  top  management  in  both  of  the  above  cate- 
gories in  "top  level"  detail.  These  plans  will  be  passed  on  to 
the  second  tier  managers  to  continue  planning  at  the  second  level 
of  detail.  This  process  will  continue  all  the  way  to  the  indi- 
vidual operating  level  where  each  person  will  make  his  personal 


29 


plans  for  getting  his  work  done.  In  developing  derivative 
plans,  two  principles  should  be  observed  at  a11  subordinate 
levels: 

Contribution  to  Objectives  Principle 

Efficiency  of  Plans  Principle 

Contribution  to  Objectives  Implies  that  every  plan  should 
provide  a positive  contribution  to  or  show  some  positive 
relationship  to  organization  objectives  at  the  next  higher 
level.  Efficiency  of  Plans  Implies  that  Implementation  of 
the  plans  will  produce  results  which  contribute  to  the  objec- 
tives with  a minimum  of  unnecessary  or  counterproductive  ef- 
fort. Another  way  to  say  It  Is  that  the  value  of  the  results 
of  the  plans  should  exceed  the  cost  of  the  effort  needed  to 
Implement  them.  Finally,  plans  need  periodic  re-evaluation, 
at  least  annually,  to  reaffirm  both  their  contributions  and 
efficiency. 

In  Air  Force  planning,  many  specific  policies  and  techniques 
are  advocated  and  directed.  These  Include  cost  effective- 
ness analysis,  life  cycle  costing,  economic  analysis,  program 
documentation  and  review,  and  numerous  others  prescribed  In 
the  regulations  and  manuals  on  program  management,  systems 
engineering,  test  and  evaluation.  Integrated  logistics  support, 
and  the  programming,  planning,  and  budgeting  cycle. 

ORGANIZING 


The  management  function  of 
organizing  has  the  fundamen- 
tal purpose  of  establishing 
a structure  of  functional 
activities  and  their  rela- 
tionships to  serve  as  a co- 
ordinating framework  In  which 
to  operate.  This  Is  accom- 
plished by  first  listing  all 
of  the  activities  which  must 
be  performed  In  order  to  ac- 
complish the  organization's 
objectives.  Then  these  ac- 
tivities are  grouped  accord- 
ing to  some  predetermined 
logic.  We  might,  for  In- 
stance, design  the  organiza- 
tional structure  by  grouping 
all  activities  that  require 
similar  skills,  or  we  might 
group  them  according  to  the 
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geographic  location  where  they  are  performed,  or  we  might 
use  a time  base  and  group  activities  In  their  chronological 
sequence.  All  of  these  and  many  others  have  been  used  suc- 
cessfully. 

The  next  major  step  In  organizing  Is  to  tie  these  grouped 
activities  together  In  a network  which  clearly  delineates 
their  relationships  to  each  other.  Inherent  In  this  pro- 
cess Is  the  Identification  of  an  individual  who  represents 
each  grouped  activity  and  who  must  be  delegated  sufficient 
authority  to  effectively  coordinate  the  activities  within 
his  aroup.  Furthermore,  there  must  be  hierarchical  (up- 
down)  and  lateral  relationships  between  groups,  and  again 
there  must  be  delegation  of  authority  to  Individuals  who 
can  effectively  coordinate  the  activities  between  these 
groups.  When  these  relationships  are  graphed  In  the  form 
of  horizontal  and  vertical  groups  of  authority  relationships, 
the  result  Is  the  conventional  organization  chart.  In  addi- 
tion to  the  chart,  written  statements  of  group  functions, 
and  Individual  Jobs,  responsibilities,  and  authority  will 
complete  the  organization  picture.  Built  Into  these  state- 
ments of  jobs  and  functions  must  be  the  general  procedures 
for  coordinating  all  the  activities  within  and  between  groups. 

As  an  example,  consider  the  organization  of  a typical  System 
Program  Office  (SPO).  The  overall  objective  of  the  organiza- 
tion Is  to  Implement  the  timely  delivery  of  systems  meeting 
defined  operational  requirements  within  the  constraints  of 
available  resources.  In  order  to  achieve  the  overall  objec- 
tive It  Is  clearly  necessary  to  establish  a set  of  coordinated 
and  specific  sub-objectives.  These  specific  sub-objectives 
are  assigned  to  functional  groupings  within  the  hierarchy  of 
the  SPO  organization.  Thus  starting  with  the  program  defini- 
tion we  typically  have  functional  activity  groupings  responsi- 
ble for  budgets  and  schedules,  configuration  of  hardware,  en- 
gineering, operations  and  testing,  and  procurement.  Occasion- 
ally, some  program  activities  normally  Included  In  one  or  the 
other  of  one  of  these  groupings  may  be  singled  out  and  be  sep- 
arately Identified.  Thus  we  find  some  SPOs  which  Include  or- 
ganizational units  for  systems  safety,  environmental  protection, 
reliability,  maintainability,  quality  control,  etc.  The  extent 
to  which  this  Is  done  depends  largely  on  the  scope  and  impor- 
tance of  these  particular  activities  In  relation  to  the  objec- 
tives of  the  program. 

The  program  manager  has  the  responsibility  to  coordinate  these 
grouped  activities  In  such  a manner  that  they  support  and  en- 
hance each  other.  In  other  words  he  facilitates  the  accomplish- 
ment of  the  SPO's  overall  objective  by  generating  an  organizational 
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climate  In  which  each  functional  activity  accomplishes  its 
assigned  tasks.  The  program  manager  is  assisted  in  the 
performance  of  his  logistics  duties  by  the  Deputy  Program 
Manager  for  Logistics  (DPMI). 

In  an  SPO  the  R&M  manager  will  typically  be  located  in 
Engineering.  Depending  upon  the  size  and  scope  of  the 
program  he  may  have  a large  or  small  staff  of  R&M  engineers 
assigned  to  him.  In  some  instances  he  may  be  operating  as 
an  individual.  However,  regardless  of  the  size  of  the  R&M 
function,  in  his  day-to-day  activities  he  will  interface 
with  other  functions  within  the  SPO.  Typically,  he  either 
seeks  or  provides  information  for  prudent  decision  making. 

In  doing  this  he  communicates  freely  across  the  hierarchical 
structure  of  the  SPO  or  other  staff  levels  as  appropriate. 
The  R&M  manager's  output  is  in  the  form  of  advice  to  other 
SPO  engineers  and  recommendations  to  the  program  manager 
who  has  ultimate  responsibility  for  the  execution  of  the 
program. 

In  summary,  when  the  management  function  of  organizing  is 
complete,  the  following  results  will  be  available: 

Organizational  chart 
Activity  function  statements 
Job  descriptions 
Authority  delegations 
Coordinating  procedures 


The  next  step  is  to  man  the  organization  with  appropriate  people. 


MANNING 


Manning  or  "staffing"  as 
it  is  more  commonly 
called  in  the  management 
literature,  is  the  man- 
agement function  of  re- 
cruiting, selecting, 
placing,  training  and 
appraising  qualified 
people  to  fulfill  the 
duties  which  have  been 
defined  through  the 
processes  of  planning 
and  organizing. 


The  obvious  challenge  Is  to  pick  the  right  person  for  the 
right  job.  This  Is  not  an  easy  task,  and  requires  re- 
cruiters with  not  only  a good  understanding  of  the  jobs 
to  be  performed,  but  a keen  Insight  Into  the  human  fact- 
ors associated  with  those  jobs.  In  fact.  It  Is  a good 
Idea  to  attach  to  each  job  description  a list  of  the  per- 
sonality attributes  most  appropriate  for  the  position. 

People  tend  to  be  loners,  social  groupers,  detail  lovers, 
gregarious  talkers,  movers,  sitters,  thinkers,  actors, 
drivers,  followers,  writers,  organizers,  leaders,  strat- 
egists, generalists,  specialists,  or  some  complex  combin- 
ation of  these  traits  and  many  more.  The  recruiter's  job 
Is  to  sniff  out  the  dominant  traits  and  skills  of  each 
person,  and  to  judge  whether  that  person  has  a good  poten- 
tial for  effectively  performing  the  duties  of  the  job. 

Once  an  Individual  has  been  hired  or  assigned  to  a job, 
there  Is  a continuing  need  to  train  him  and  develop  his 
abilities  to  function  efficiently.  This  may  Include  train- 
ing In  technical  areas,  communications  techniques,  manage- 
ment skills  and  Interpersonal  relations.  Appraisal  of  the 
Individual's  job  performance  can  also  be  considered  a de- 
velopment technique. 

A periodic  function  of  manning  Is  appraisal  of  the  Indi- 
vidual's performance.  While  most  appraisal  methods  tend 
to  be  based  on  subjective  judgments,  it  Is  far  better  to 
base  appraisal  on  the  Individual's  achievement  of  quanti- 
fiable objectives  that  he  has  agreed  In  advance  are  reason- 
able for  him  to  pursue.  This  method  of  appraisal  serves  to 
keep  everyone  objectives  oriented,  and  will  tend  to  play 
down  subjective  personality  judgments  hinged  on  whether  or 
not  the  person  happens  to  be  likeable.  The  use  of  an  ap- 
praisal based  on  quantifiable  objectives  also  makes  dis- 
cussion of  the  person's  appraisal  much  more  productive, 
since  It  Is  easier  to  focus  on  the  objectives  and  how  well 
they  were  or  were  not  met  along  with  the  various  reasons 
why.  You  might  also  get  some  good  feedback  responses  that 
would  be  useful  for  Improving  the  organization  structure  or 
procedures.  If,  on  the  other  hand,  you  get  focused  on  the 
Individual's  personality  traits.  It  Is  very  easy  to  turn  off 
useful  communications  or  to  degenerate  the  discussion  Into 
an  emotional  exchange  that  would  be  counterproductive.  Ap- 
praisals based  on  judgment  of  subjective  personality  traits 
also  have  a strong  tendency  to  get  Inflated  If  the  subjects 
of  the  appraisals  have  access  to  them. 
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While  appraisal  based  on  achievement  of  quantified  ob- 
jectives sounds  good.  It  Is  easier  said  than  done.  Many 
jobs  do  not  have  objectives  which  are  easy  to  quantlfy-- 
take  a receptionist  or  recruiter  for  example. 

To  summarize,  manning  consists  of: 

Selecting  and  placing  qualified  people  to  perform 
the  defined  jobs 

Training  and  developing  their  skills 

Appraising  their  performance  and  making  appropriate 
changes  In  job  assignments,  training  or  development 

LEADING 


The  management  function  of 
leading  consists  of  set- 
ting  the  organization 
activities  Into  motion 

towards  the  objectives  em-  V ^ 

ploying  established  re-  \ 

sources,  procedures,  poll- 
cles,  organization  and  A 

schedules,  and  Inspiring  ^ i 

enthusiastic  participa- 

tion  by  subordinates  and  M/mf 

associates.  This  Is  ac- 

compllshed  through  the  

use  of  orders,  directives,  KinniiTTtfir^  _ — ^ / [ 

Instructions,  explanations,  - ■ r.i.TTTrririMii  ir^  / > — 
persuasion,  encouragement,  ^ 

motivation,  rewards,  pen-  

altles,  and  many  more. 

From  these  It  Is  obvious 

that  the  leader  must  be  skilled  In  using  the  graphic  and  verbal 
forms  of  communications.  He  has  to  be  a good  coordinator  and 
motivator.  He  has  to  work  with  people  and  Inspire  people  to  work 
with  each  other. 

A first  and  fundamental  activity  for  effective  leading  Is  to  make 
organizational  objectives  and  priorities  very  clear  to  all  subor- 
dinates. Everyone  should  be  able  to  visualize  how  his  duties  con- 
tribute to  accomplishment  of  those  objectives,  and  he  should  be 
encouraged  to  suggest  changes  In  his  duties  If  his  duties  Involve 
some  counterproductive  actions.  He  cannot  make  these  suggestions 
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unless  the  objectives  are  explicit,  tangible  things  which 
he  can  measure  or  observe.  That  is  why  derivative  plans 
must  be  developed  which  provide  objectives  and  priorities 
for  all  levels  to  use  down  through  the  organization. 

A second  fundamental  activity  in  leading  is  to  also  make 
very  clear,  all  rules,  policies  and  procedures  applicable 
to  the  jobs  of  subordinates,  so  they  will  know  how  to  get 
things  done.  These  policies  and  procedures  result  from 
the  planning  process,  and  the  task  in  leading  is  to  ex- 
plain their  use  and  work  out  any  bugs  discovered  through 
their  use. 

The  leader  must  then  initiate  action,  or  set  the  organiza- 
tion wheels  into  motion,  by  issuing  instructions  in  one 
form  or  another.  It  may  be  at  this  point  that  the  manager 
develops  his  final  sets  of  derivative  plans,  objectives, 
schedules,  resources,  tasks  for  his  subordinate  to  imple- 
ment, and  includes  these  in  his  initiating  instructions. 

These  instructions,  together  with  procedures, should  insure 
that  activities  are  coordinated  (supplement  or  compliment 
one  another). 

The  last  and  most  challenging  task  of  leading  is  to  provide 
motivation  to  perform.  This  is  meant  to  go  well  beyond  the 
customary  rumuneration  of  most  jobs.  Motivation  involves 
setting  a good  example,  making  work  seem  enjoyable  and  sat- 
isfying, setting  high  standards  of  integrity  and  performance, 
praising  good  work  (in  private  and  public),  showing  how  to 
correct  deficiencies  (always  in  private),  handling  human  re- 
lations problems  with  fairness  and  respect  for  the  dignity  of 
the  individual,  promoting  an  atmosphere  of  open  communications 
in  all  organization  directions,  and  many  more.  If  all  of  these 
leadership  actions  are  handled  skillfully,  we  can  succeed  even 
without  charisma. 

In  summary  then,  leading  involves: 

Explaining  and  publicizing  objectives 
Clarifying  policies  and  procedures 

t 

Initiating  action  with  explicit  instructions 
Promoting  coordination  of  activities 
Motivating  through  a variety  of  techniques 
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CONTROUING 
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Control  requires  the  measurement  of 
accomplishments,  comparing  them  with 
preplanned  standards,  and  taking  cor- 
rective action  if  deviations  are  un- 
acceptable. Control  procedures  should 
be  planned  for  all  organization  activ- 
ities and  should  be  formal  for  critical 
activities,  but  can  be  informal  for 
lesser  activities.  The  appropriate 
degree  of  control  depends  on  the  degree 
to  which  the  activity  impacts  organiza- 
tion objectives.  Control  is  usually 
applied  to  resource  expenditures,  prod- 
uct quality  and  quantity,  certain  services  rendered,  task 
accomplishment  versus  schedules,  and  compliance  with  orga- 
nization procedures. 


! 
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Controls  should  focus  on  a relatively  few  strategic  param- 
eters or  Indicators  of  progress  towards  objectives  of  the 
activity,  and  should  avoid  unnecessary  or  redundant  measures. 
Controlling  requires  gathering  data,  analyzing  and  summariz- 
ing it,  and  presenting  it  in  an  easy-to-comprehend  form. 

This  can  be  a costly  administrative  task,  and  prudence  is 
advisable. 
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The  gathering,  analysis  and  presentation  of  control  data  , 

should  be  planned  at  an  appropriate  frequency  and  on  a reason-  ^ 

able  scale.  For  example,  if  we  had  a $50,000  monthly  budget 
which  is  capable  of  varying  by  only  $10,000  in  any  month  (due 
to  mostly  fixed  costs),  it  would  be  senseless  to  review  expend- 
itures to  the  nearest  dollar  on  a daily  basis.  A weekly  review, 
rounded  off  to  the  nearest  hundred  dollars  would  be  more  rea- 
sonable. 

If  performance  makes  unacceptable  deviations  from  the  standards, 
corrective  action  must  be  quick  enough  to  do  some  good,  and  also 
should  be  tailored  to  reduce  the  chances  of  deviations  in  the 
future.  Obviously,  corrective  action  is  implemented  by  the  man- 
ager responsible  for  the  activity  being  controlled. 

To  minimize  the  need  for  corrective  action,  the  organization 
should  plan  procedures  and  policies  which  motivate  people  to- 
wards organization  objectives  by  creating  incentives  that  ap- 
peal to  individual  human  needs  and  drives.  Motivation  of  the 
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individual  to  control  his  own  performance  toward  organiza- 
tion objectives  is  a far  superior  approach  to  external  con- 
trol by  superiors.  That  is  because  external  conf^ol  can 
easily  lead  to  a feeling  of  mutual  distrust  between  supe- 
riors and  subordinates,  resentment,  resistance  and  a ten- 
dency to  withhold,  cover-up  or  distort  information  that  may 
put  the  individual  in  an  unfavorable  light.  External  con- 
trols can  also  be  expensive  in  terms  of  administrative 
burden,  and  can  easily  degenerate  into  counterproductive 
effort  if  not  carefully  conceived  and  prudently  applied. 

Some  examples  of  procedures  or  policies  which  have  a self- 
regulating  affect  on  organized  achievement  are  listed  be- 
low. (Some  of  these  might  be  considered  motivations  under 
the  preceding  heading  of  "leading,"  but  that  is  unimportant. 
Their  effect  is  to  reduce  the  need  for  external  control  cor- 
rections in  the  organization). 

a.  Offer  monetary  incentives  for  achieving  quantity, 
quality  or  schedule  goals. 

b.  Appeal  to  personal  pride  in  workmanship  by  imprint- 
ing the  worker's  name  on  the  product. 

c.  Appeal  to  pride  of  authorship  and  the  desire  for  per- 
sonal fulfillment  by  asking  employees  to  derive  their  own 
objectives  and  goals  from  higher  level  objectives,  and  later 
report  on  their  accomplishment  of  them, 

d.  Appeal  to  the  desire  of  the  individual  to  have  per- 
sonal control  over  his  activities,  by  asking  groups  to  develop 
their  own  operating  procedures  and  later  report  on  their 
effectiveness. 

e.  Appeal  to  the  spirit  of  group  competition  by  reward- 
ing groups  with  the  best  performance. 

f.  Appeal  to  the  sense  of  pride  in  the  organization  by 
setting  high  standards  of  performance  and  making  it  clear 
what  is  expected  of  its  members.  However,  even  with  these 
kinds  of  incentives  for  achievement,  organization  progress 
towards  goals  still  need  to  be  measured,  but  the  need  for 
corrective  actions  by  management  should  be  significantly  re- 
duced and  overall  organization  efficiency  increased. 

External  management  control  techniques  may  range'  from  a 
weekly  staff  meeting  with  verbal  progress  reports,  to  elab- 
orate data  gathering,  analysis  and  charting  schemes  combined 
with  feedback  Instructions  to  the  managers  of  functional 
groups  being  controlled. 
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Suirmarizing,  management  control  requires: 
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Selection  of  activity  performance  parameters  worthy 
of  tracking 

Setting  standards  of  performance 

Choosing  appropriate  scale  and  frequency  of  obser- 
vations 

Comparing  performance  with  standards 

Taking  timely  action  to  correct  deviations  and  to 
reduce  chances  of  repetition 

Search  for  self-control  ling  procedures  founded  on 
individual  incentives  and  motivations 

PROGRAM  MANAGEMENT 


In  Air  Force  terminology,  program  management  refers  to  the 
management  processes  involved  in  developing,  testing  and  pro- 
ducing new  military  systems.  The  Program  Manager  is  given 
broad  responsibilities  for  producing  a good  system  and  is 
delegated  substantial  authority  to  make  the  decisions  needed 
to  fulfill  those  responsibilities.  The  objective  is  to  "de- 
centralize" program  management  as  far  as  it  is  prudent  to  do 
so,  and  focus  the  responsibility  and  day-to-day  decision  mak- 
ing in  the  program  office. 

The  Department  of  Defense  (DoD)  and  Air  Force  policies  govern- 
ing the  Program  Manager's  role  are  explained  in  DoD  Directive 
5000.1  which  is  an  attachment  to  Air  Force  Regulation  800-2, 
Program  Management.  The  regulation  outlines  the  major  manage- 
ment responsibilities  of  the  Program  Manager,  with  emphasis  on 
general  planning  and  reporting  requirements,  and  his  decision- 
making authority.  Anyone  involved  with  a system  acquisition 
program  must  be  thoroughly  familiar  with  AFR  800-2. 


THE  DOMAIN  OF  PROGRAM  MANAGEMENT 
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R&M  MANAGEMENT 


The  R&M  manager's  job  Is  not  a simple  one  since  most  of 
the  people  he  must  Influence  do  not  work  for  him.  He 
must  motivate  many  people  In  other  elements  of  the  or- 
ganization. Therefore,  while  he  Is  first  of  all  a man- 
ager, he  needs  to  have  some  knowledge  of  the  various 
professional  specialties  Involved  In  the  program.  These 
Include  systems  and  equipment  engineering,  contract  law 
and  regulations,  configuration  control,  economic  decision 
making,  and  perhaps  others  like  aeromedicine  or  psychology. 
The  R&M  manager  has  a formidable  task  of  leading  the  R&M 
program  along  a hazardous  path  of  pitfalls  scattered 
throughout  the  development  cycle.  He  must  be  a manager 
skilled  In  applying  the  management  principles  described 
In  this  chapter. 

One  purpose  of  this  Reliability  and  Maintainability  Man- 
agement Guide  Is  to  clarify  the  technical  and  management 
pitfalls  In  R&M,  and  to  show  how  to  formulate  a success- 
ful R&M  program  based  on  sound  engineering  and  management 
principles. 

SYSTEMS  ENGINEERING 

The  term  "engineering" 
usually  means  the  appli- 
cation of  traditional 
technical  and  scientific 
skills  (mathematics, 
physics,  chemistry,  etc.) 
to  conceive,  design, 
construct,  test  and 
produce  new  things. 

"Management,"  on  the 
other  hand.  Is  the  ap- 
plication of  human  be- 
havior principles  and  in- 
sights (psychology,  phil- 
osophy, physiology,  etc.) 
to  plan,  organize,  man, 
lead,  and  control  any  human 
endeavor.  Clearly,  manage- 
ment Is  Involved  In  the  or- 
derly activities  of  any  group,  but  engineering  Is  not. 
Whether  we  talk  about  engineering,  management,  or  some  other 
disciplines,  communication  Is  always  the  hub  of  organized 
activity. 
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"Systems  engineering,"  as  the  term  is  used  in  the  Air  Force, 
is  a more  general  term  that  encompasses  both  engineering  and 
management  aspects.  It  means  that  the  military  system  must 
be  conceived,  designed,  constructed,  tested,  and  produced, 
taking  into  account  the  performance  and  economic  trade-offs 
involved  across  all  functional  and  support  elements  of  the 
system  and  over  the  entire  life  span  of  its  use.  That  is, 
systems  engineering  is  expected  to  yield  the  most  cost  ef- 
fective system  considering  operational  performance,  produci- 
bility,  and  supportabil ity . To  achieve  this  goal,  various 
management  planning,  organizing,  leading,  and  controlling 
policies  and  procedures  are  prescribed.  Air  Force  Regulation 
800-3,  Engineering  for  Defense  Systems,  explains  what  those 
policies  and  general  procedures  are.  The  Program  Manager  is 
responsible  for  seeing  that  systems  engineering  is  employed 
in  his  program. 

This  introduces  us  to  the  elements  of  reliability  and  main- 
tainability (R&M)  programs  explained  in  the  next  two  chapters, 
since  these  R&M  programs  are  part  of  the  systems  engineering 
process. 

As  the  reader  will  notice,  R&M  programs  consist  of  a mix  of 
engineering  and  management  elements  which  mesh  with  other 
systems  engineering  tasks  to  support  R&M  objectives.  (When 
reading  the  following  chapters,  it  might  be  instructive  to 
note  which  of  the  R&M  program  elements  are  basically  managerial 
and  which  elements  are  fundamentally  technical  or  engineering). 

The  remainder  of  this  guidebook  gets  into  the  specifics  of  R&M 
program  management,  with  enough  technical  description  to  make 
the  program  elements  tangible  and  meaningful. 


40 


Chapter  4 

ELEMENTS  OF  A RELIABILITY  PROGRAM 


INTRODUCTION 

To  successfully  reach  its  goal,  a reliability  program  requires 
coordinated  performance  of  a series  of  tasks  by  managers  and 
technical  specialists.  This  series  begins  with  the  first  con- 
ceptual studies  of  the  new  system,  continues  through  production, 
and  ends  only  when  the  system  is  phased  out  of  use.  The  procur- 
ing agency  (AFSC),  the  manufacturer  (contractor),  the  user  (op- 
erational command),  and  his  support  agency  (AFLC),  all  have  re- 
sponsibilities in  this  chain  of  events. 

The  staff  R&M  manager  needs  a detailed  understanding  of  the  re- 
liability engineering  and  management  tasks,  including  who  does 
them  and  when  they  must  be  done.  He  must  understand  the  technical 
terminology  of  reliability  engineering  and  how  to  specify  relia- 
bility performance,  and  he  must  know  the  engineering  data  required 
to  track  a contractor's  R&M  program.  He  should  understand  the  rel- 
ative importance  of  the  various  activities  and  the  possible  conse- 
quences of  skipping  or  curtailing  them.  He  should  recognize  major 
options  with  corresponding  costs  and  risks.  He  should  know  where 
to  get  additional  technical  people  for  advice.  This  short  chapter 
is  written  to  fulfill  those  needs. 

STANDARD  RELIABILITY  PROGRAM  ELEMENTS 
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MIL-STD-785 


The  chart  which  follows  lists 
the  elements  of  a hardware  re- 
liability program  and  shows  the 
importance  of  each  element  dur- 
ing the  life  cycle  phases  of 
development.  This  list  general- 
ly follows  the  outline  of  MIL- 
STD-785,  but  with  some  changes 
to  aid  continuity  of  the  dis- 
cussion. MIL-STD-785  is  the 
basic  standard  for  planning 
reliability  programs  for  DoD 
development  and  production  con- 
tracts and  gives  guidelines  for 
preparing  a reliability  program 
plan.  However,  the  application 
of  MIL-STD-785  provisions  is  subject  to  the  discretion  of  the  pro- 
curing authority.  To  intelligently  exercise  this  discretion,  the 
procuring  authority  needs  expert  advice  from  someone  with  R&M 


Reliability  Program 
for 

Systems  and  Equipment 
Development  and  Production 
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RELIABILITY  PROGRAM  ELEMENTS 


engineering  and  management  experience.  The  procuring  authority 
is  usually  advised  by  the  project  manager  and  sometimes  is  the 
project  manager.  However,  the  project  manager  is  rarely  an  R&M 
engineer  and  therefore  he  also  needs  the  advice  of  R&M  special- 
ists. The  explanations  which  follow  should  aid  communication  be- 
tween manager  and  advisor,  and  help  the  R&M  advisor  provide  the 
correct  advice  to  the  project  manager. 

First,  a definition  of  reliability  may  be  helpful.  Reliability 
is  a performance  attribute  of  an  item  and  defines  its  ability 
to  properly  function  under  specified  conditions  for  a certain 
period  of  time  or  a certain  number  of  operating  cycles.  Relia- 
bility can  be  quantified  as  a probability.  For  example,  we  could 
say  that  the  probability  is  .90  that  a Model -A  machine  gun  will 
fire  successfully  for  60  seconds  in  rainfall  rate  of  one  inch  per 
hour.  Reliability  can  also  be  quantified  as  a Mean-Time-Between- 
Failure  (MTBF).  That  is,  Model-A  machine  guns  have  an  MTBF  of 
570  seconds  firing  over  60  second  intervals  in  a rainfall  rate  of 
one  inch  per  hour.  These  probability  and  MTBF  numbers  are  math- 
ematically related  to  one  another  and  are  called  "reliability 
figures  of  merit,"  or  simply  "reliability  figures"  to  be  more 
brief.  Definitions  of  these  and  other  R&M  terms  can  be  found  in 
MIL-STD-721,  Definitions  of  Effectiveness  Terms  for  Reliability, 
Maintainability,  Human  Factors,  and  Safety.  The  mathematical  re- 
lationships can  be  found  in  R&M  engineering  books. 

Before  explaining  the  separate  reliability  program  elements  listed 
on  the  chart,  some  general  comments  about  the  chart  would  be  help- 
ful . 

The  relative  importance  rating  of  the  elements  represents  the  sub- 
jective judgment  of  the  authors  based  on  years  of  experience,  and 
is  meant  to  apply  to  the  "average"  development  program.  The  chart 
is  designed  to  give  the  R&M  manager  an  overview  or  feeling  for  the 
average  situation.  Every  development  program  is  different  and  the 
reliability  program  to  go  with  it  must  be  tailored  to  specific  needs. 
This  tailoring  must  be  done  by  reliability  specialists  working  for 
the  Air  Force  program  manager. 

Only  the  first  conceptual  study  contract  milestone  is  shown.  Work 
shown  to  the  left  is  Air  Force  homework  leading  to  the  first  state- 
ment of  work.  These  conceptual  studies  and  subsequent  contractual 
work  will  lead  to  more  and  more  specific  design  and  acceptance  spec- 
ifications until  the  production  contract  is  solicited.  Several  con- 
tracts may  be  used  between  the  conceptual  phase  and  the  production 
phase,  with  initiation  occurring  at  the  beginning  of  each  phase  fol- 
lowing go-ahead  decisions  from  Air  Force  and  Department  of  Defense 
management.  On  some  programs  many  of  the  conceptual  and  validation 
phase  tasks  are  accomplished  by  Government  planning  organizations 
or  by  the  program  office.  This  does  not  affect  the  relative  impor- 
tance of  the  tasks  or  the  necessity  for  having  them  accomplished  at 
the  proper  time.  Therefore,  the  R&M  manager  must  review  all  the 
required  tasks,  and  for  those  which  are  not  to  be  accomplished 
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contractually  insure  appropriate  Government  organizations  have  been 
designated  to  accomplish  them. 


1 


DEFINITION  OF  REALISTIC  REQUIREMENTS 

The  first  and  most  important 
task  in  a reliability  program  is 
selection  or  definition  of  real- 
istic requirements.  This  is  a pro- 
curing activity  task.  While  exe- 
cution of  this  task  requires  the 
knowledge  of  reliability  engineers 
it  is  closely  tied  to  the  managerial 
function  of  planning.  The  basis  for 
the  selected  requirements  should  be 
questioned  by  managers  and  planners 
at  all  organization  levels,  and  that 
includes  the  user,  supporter  and 
contractor,  as  well  as  AFSC.  These 
requirements  are  the  objective  of 
the  reliability  program,  and  hardly 
anything  is  more  wasteful  or  dis- 
rupting than  to  strive  for  the  wrong 
objectives.  Managerial  scrutiny  of 
the  requirements  should  begin  early 
in  the  conceptual  phase  and  continue 
through  validation  and  into  full  scale 
development.  As  the  chart  shows,  the 
final  setting  of  realistic  require- 
ments is  critical  at  the  beginning  of 
full  scale  development. 

The  word  "realistic"  needs  emphasis.  Whether  a complex  or  simple 
system,  the  realism  of  the  reliability  requirement  will  determine 
much  of  the  long-term  success  of  the  R&M  program.  Too  high  a re- 
liability figure  can  lead  to  excessive  costs  in  attempting  to  a- 
chieve  it,  program  disruptions  when  it  becomes  evident  that  the 
figure  cannot  be  met,  litigation  based  on  claims  of  impossibility, 
fouled  plans  for  logistic  support,  and  finally,  possible  compro- 
mise to  an  unnecessarily  low  reliability  figure  because  of  schedule 
and  cost  pressures  late  in  the  program.  On  the  other  hand,  too  low 
a requirement  at  the  beginning  usually  results  in  insufficient 
R&M  program  emphasis  with  the  specified  requirement  being  easily 
met,  and  loss  of  an  opportunity  to  get  higher  reliability  and 
lower  support  costs. 

The  procuring  activity  must  define  the  requirement  with  inputs 
from  the  using  command  and  AFLC,  since  the  final  requirement 
must  be  operationally  adequate  and  logistically  supportable. 

(See  AFR  66-14,  and  AFSC  Supplement  1,  Equipment  Maintenance 
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Policies,  Objectives,  and  Responsibilities,  and  APR  80-5).  A 
tradeoff  analysis  should  be  performed  using  systems  cost  effec- 
tiveness analysis  and  life  cycle  cost  techniques,  since  relia- 
bility is  a system  performance  parameter  which  strongly  affects 
system  effectiveness  and  life  cycle  cost.  In  these  analyses, 
reliability  should  be  varied  over  a reasonable  range  of  values  to 
establish  the  sensitivity  of  system  effectiveness  and  life  cycle 
cost  to  the  reliability  parameter.  These  sensitivity  curves  should 
then  be  used,  along  with  the  projections  of  reliability  engineers, 
to  select  reliability  requirements  which  strike  a reasonable  bal- 
ance between  operational  and  logistic  needs,  and  the  avaiUble 
technology  to  fulfill  those  needs. 

The  reliability  which  can  be  reasonably  achieved  for  a particular 
type  equipment  using  available  technology  must  be  estimated  by 
reliability  specialists.  These  specialists  have  a variety  of 
ways  to  come  up  with  reasonable  estimates,  and  all  are  based  on 
the  use  of  historical  data  in  one  way  or  another.  The  most  direct 
approach  is  to  survey  the  reliability  achieved  by  similar  types 
of  equipment  in  field  use,  examine  the  caliber  of  each  reliability 
program  used  during  development  of  those  equipments,  make  adjust- 
ments for  technology  progress  since  those  equipments  were  developed, 
adjust  for  complexity  differences  between  the  old  equipments  and 
the  new,  and  finally  adjust  for  differences  in  reliability  program 
emphasis  planned  for  the  new  equipment.  This  is  not  an  easy  task 
since  it  depends  on  finding  good  historical  records.  Furthermore, 
it  is  important  that  the  proper  comparisons  be  made.  A histori- 
cal MTBF  using  field  data  is  a different  measure  than  a MIL- 
STD-781  specified  MTBF.  Methods  of  estimation  with  a more  analy- 
tical flavor  are  based  on  expected  numbers  of  part  types  in  the 
new  equipment,  the  parts  quality  levels  Which  should  be  available, 
the  equipment  configuration  anticipated,  the  environment  in  which 
it  will  be  used,  and  mathematical  computations  which  take  these 
parameters  into  account.  Remember  that  even  in  those  systems  touted 
as  breakthroughs,  state-of-the-art,  or  "all  new  and  different"  in 
concept,  the  really  new  things  are  only  a small  part  of  the  total 
and  the  rest  of  it  is  conventional  hardware. 

In  addition,  the  Reliability  manager  must  recognize  the  difference 
between  Reliability  terms  used  in  reporting  within  the  Air  Force  and 
terms  used  in  the  contract.  These  terms  may  differ,  but  the 
contractual  terms  must  be  translatable  into  the  reporting  terms. 

AFR  80-5  provides  the  standard  reliability  terms  for  reporting.  AFSC 
Supplement  1 to  AFR  80-5  (Nov  1978  draft  in  coordination  as  this  is 
written,  presumably  in  effect  as  you  read  it),  also  provides  guidance 
for  converting  AFLC  data  (from  the  AFLC  DO-56  Product  Performance 
System)  to  standard  reporting  terms  by  proper  sorting  of  how  mal- 
function codes  and  action  taken  codes.  With  this  guidance  the  AFSC 
program  manager  can  obtain  from  AFLC  data  formatted  to  minimize 
definitional  differences. 
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Summarizing  the  above,  reliability  requirements  are  developed 
from  systems  effectiveness  and  life  cycle  cost  studies,  coupled 
with  projections  on  what  is  reasonable  to  achieve  using  known 
technology.  To  determine  realistic  requirements,  the  assistance 
of  experienced  reliability  specialists  is  necessary.  While  not 
a trivial  problem,  methods  and  data  exist  to  solve  it.  The  pro- 
curing activity  is  responsible  for  this  task,  but  the  desires  of 
the  user  and  support  agency  must  be  considered.  Significance  of 
this  task  ranges  from  very  important  to  critical  depending  upon 
development  phase. 

DEVELOPMENT  OF  A RELIABILITY  MODEL 

A reliability  model  is  a mathematical  equation  which  defines 
the  relationship  between  the  failure  rate  of  an  assembly  (equip- 
ment, or  system)  and  the  failure  rates  of  all  the  parts  which 
make  up  the  assembly.  Each  part,  in  turn,  has  a reliability  model 
which  relates  its  failure  rate  to  part  quality,  operating  stress 
or  derating  level,  and  the  physical  environment  in  which  the  part 
is  to  function. 

The  reliability  model  of  the  assembly  is  derived  by  reliability 
engineers  from  functional  diagrams,  circuit  diagrams,  or  detail 
design  drawings  of  the  assembly.  The  result  is  a flow  diagram 
which  depicts  the  series  or  parallel  interdependencies  between 
all  the  parts.  This  diagram  is  then  expressed  as  a mathematical 
equation  which  becomes  the  reliability  model. 

Failure  models  for  the  separate  parts  are  developed  through  anal- 
ysis of  laboratory  test  and  field  data,  and  detailed  studies  of 
the  physical  mechanisms  which  give  rise  to  part  failure.  This  is 
a very  complex  business  done  only  by  Government  and  commercial  lab- 
oratories that  have  specialized  equipment  and  people.  The  models  are 
compiled  in  MIL-HDBK-217,  Reliability  Prediction  of  Electronic 
Equi pment. 

The  resulting  reliability  model  of  an  assembly,  equipment  or  system 
is  the  analytical  basis  for  making  reliability  predictions.  While 
this  model  may  be  rather  crude  during  the  conceptual  phase,  it  will 
be  expanded  and  refined  as  more  system  details  are  evolved  in  vali- 
dation and  full  scale  development.  The  model  must  be  good,  other- 
wise the  very  important  reliability  predictions  to  be  tracked  later 
by  management  will  be  misleading. 

The  project  manager  should  be  certain  that  the  agency  (usually  the 
equipment  contractor)  who  develops  this  model  does  it  correctly. 

This  model  development  should  be  reviewed  in  detail  for  the  project 
manager  by  technical  R&M  specialists  who  are  independent  of  the 
contractor. 


RELIABILITY  PREDICTIONS 


The  reliability  of  an  assembly, 
equipment  or  system  is  predicated 
by  inserting  into  its  reliability 
model  the  failure  rates  of  the  sep- 
arate parts.  These  part  failure 
rates  are  obtained  from  the  parts 
reliability  models  by  considering 
the  quality  class  of  each  part, 
its  proposed  operating  stress  level 
(or  derating)  and  the  physical  envir- 
onment in  which  it  will  operate. 
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During  the  conceptual  phase,  predic- 
tions will  be  less  accurate,  because 
of  incomplete  system  and  parts  data 
obtained  from  estimates  or  design 
specifications.  As  equipment  design 

progresses  during  validation  and  full  scale  development,  predic- 
tions will  improve  and  become  the  quantitative  backbone  of  the 
reliability  program.  These  predictions  should  be  compared  with 
reliability  requirements  and  test  results  throughout  the  develop- 
ment phases  by  both  engineers  and  managers.  This  will  serve  to 
illuminate  reliability  program  progress  and  problem  areas,  and 
support  the  need  for  any  engineering  changes.  It  will  show  a 
pictorial,  historical  record  of  progress  that  will  be  a focal  point 
of  discussions  during  design  and  program  reviews. 
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For  engineering  details  on  systems  reliability  modeling  and 
prediction  techniques,  and  specific  models  for  predicting 
failure  rates  of  electronic  and  mechanical  parts,  consult 
MIL-HDBK-217,  Reliability  Prediction  of  Electronic  Equipment. 

RELIABILITY  APPORTIONMENT 

The  converse  of  reliability  prediction  is  reliability 
apportionment.  This  is  the  analytical  process  by  which  the 
total  system  reliability  requirement  is  apportioned  or  allo- 
cated among  the  separate  subsystems  or  equipments  which  com- 
prise the  system.  These  apportioned  reliability  figures  then 
become  the  design  requirement  for  each  subsystem.  The  sub- 
system designer  may  in  turn  further  apportion  his  requirement 
among  the  subassemblies  of  his  subsystem. 

In  application,  there  is  considerable  interplay  between  pre- 
diction, apportionment  and  system  design.  First,  the  apportion- 
ment helps  to  establish  a system  and  subsystem  design  approach 
which  should  meet  the  system  reliability  requirement.  As  detail 
design  progresses  and  parts  are  selected,  predictions  are  per- 
formed to  see  if  the  selected  design  can  actually  meet  require- 
ments. If  not,  the  design  may  be  adjusted  or  the  apportionment 
redone  to  set  more  realistic  subsystem  goals. 

Initial  apportionment  is  performed  by  the  group  responsible  for 
system  integration  (usually  a contractor)  so  that  vendors  or 
equipment  designers  can  be  given  design  requirements.  Occasion- 
ally, when  the  Air  Force  is  purchasing  individual  equipments  for 
"in-house"  integration,  the  apportionment  will  be  an  Air  Force 
responsibility. 

FAILURE  MODES  AND  EFFECTS  ANALYSIS 

A companion  effort  along  with  reliability  modeling,  prediction, 
apportionment  and  design,  is  "failure  modes  and  effects  analysis" 
(FMEA).  This  is  a review  of  the  system  design  to  identify  failure 
possibilities  so  that  they  can  be  eliminated  or  minimized  through 
corrective  design  changes  while  the  design  is  still  easily  modi- 
fied. A special  form  of  FMEA  is  the  fault  tree  analysis  used  by 
safety  engineers  to  identify  and  eliminate  possible  safety  haz- 
ards. In  more  general  reliability  engineering,  the  FMEA  can  iden- 
tify areas  where  protective  circuitry  or  structures  should  be  used 
to  prevent  the  failure  of  one  component  from  overstressing  others, 
can  identify  critical  components  whose  reliability  warrants  special 
attention,  or  can  identify  potential  adjustment  and  timing  problems, 
etc.  The  FMEA  is  also  valuable  in  maintainability  analysis  to  be 
discussed  later. 

The  FMEA  must  be  performed  by  someone  who  is  familiar  with  and 
able  to  influence  detail  design.  Hence,  it  is  usually  the 
responsibility  of  the  equipment  designer.  The  extent  of  the 
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FMEA  can  range  from  a simple  examination  of  the  system  or  ' 

equipment  functional  diagram,  to  a detailed  analysis  of  the 
design  drawings  and  schematics  considering  the  failure  rate  ' 
of  each  part  and  its  likely  mode  of  failure. 

Therefore,  the  time  and  cost  required  for  a FMEA  depend  on 
its  emphasis,  which  in  turn  depends  on  the  complexity  and 
purpose  of  the  system.  In  most  cases,  a rather  detailed  FMEA 
is  warranted,  even  though  it  is  much  more  expensive  than  the 
reliability  modeling  and  prediction  activity.  It  is  a good 
investment  since  corrective  actions  resulting  from  the  FMEA 
can  be  easily  implemented  before  design  is  frozen.  The  same 
changes  resulting  from  later  hardware  tests  will  be  much  more 
expensive  to  implement. 

DESIGN  FOR  RELIABILITY 

Design  for  reliability  is  an  omnibus  title,  carrying  a 
myriad  of  individual  elements  related  by  the  common  fact  that 
they  all  must  be  considered  by  the  designer  of  the  system, 
equipment  or  assembly  to  assure  its  reliability.  The  following 
list  includes  typical  considerations  to  illustrate  the  point. 

A nwre  complete  checklist  for  a given  item  depends  on  whether 
it  incorporates  electronics,  mechanics,  structures,  hydraulics, 
pneumatics  or  some  other  technology. 

Simplicity  of  design 

Producibility  of  design 

Use  of  Government  and  industry  standard  design 
and  layout  practices 

Use  of  redundant  or  fail-safe  designs  (use  FMEA) 

Provisions  for  optional  modes  of  operation 

Use  of  preferred  or  proven  parts  and  materials 

Selection  of  appropriate  load  or  derating  factors 
for  parts  and  materials 

Controllability  of  parts  and  materials  quality 

Future  availability  of  good  replacement  parts  and  materials 

Consideration  of  aging  or  fatigue  effects 

Consideration  of  human  factors  on  reliability  in 
manufacture,  operation  or  maintenance 

I 

Prediction  and  control  of  physical  environment 
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including  temperature,  moisture,  vibration,  shock, 
dust,  chemicals,  radiation,  ambient  pressure,  and 
electrical  interference 

The  producer  of  the  hardware  is  responsible  for  formulating 
and  carrying  out  this  activity.  The  Air  Force,  however,  is 
responsible  for  selecting  a competent  producer  and  Insuring 
that  the  contract  motivates  the  producer  to  perform  these 
design  tasks  to  the  best  of  his  ability.  The  reliability 
program  elements  to  follow  include  these  motivations. 

Aside  from  such  overhead  costs  as  reliability  training  pro- 
grams and  preparation  of  reliability  design  manuals  and  check- 
lists, it  does  not  cost  much  more  to  design  an  item  for  relia- 
bility than  to  design  it  without  consideration  of  reliability. 
Whatever  the  small  increased  cost  may  be,  it  is  well  worth  the 
investment  at  the  design  stage.  Fixing  defective  systems  at 
the  testing  stage  is  far  more  costly. 

PARTS  SELECTION 

Parts  selection  is  a critical  reliability  engineering  ele- 
ment. There  is  no  question  that  parts  quality  is  a costly  item, 
and  for  this  reason,  contractors  are  tempted  to  compromise  parts 
quality.  Yet  wisdom  says,  "You  cannot  make  a silk  purse  out  of 
a sow's  ear."  With  rare  exceptions,  you  cannot  make  a satisfac- 
tory Air  Force  system  from  commercial  grade  parts.  If  the  Air 
Force  system  is  something  like  Minuteman  or  a manned  space  system, 
even  high  quality  military  grade  parts  may  be  inadequate.  For  ex- 
ample, the  Minuteman  program  established  specially  controlled  parts 
manufacturing  lines  for  its  own  use.  A great  deal  of  effort  has 
been  expended  to  produce  high  quality  parts  for  Air  Force  use. 

While  standard  military  quality  specifications  provide  assurance 
that  parts  can  withstand  the  environmental  extremes  of  Air  Force 
use,  these  specifications  alone  do  not  assure  low  failure  rates, 
and  so  further  controls  have  been  created.  For  example,  in  the 
field  of  electronics,  the  most  successful  have  been  Established 
Reliability  (ER)  specifications.  These  specifications  require 
tests  which  verify  specified  failure  rates.  Applied  to  passive 
electronic  components,  they  have  been  in  effect  for  several  years. 
Indeed,  it  is  now  possible  to  buy  resistors  and  capacitors  to  ER 
specifications  as  cheaply  as  to  standard  military  quality  specifi- 
cations. 

For  semiconductor  electronic  devices,  "TX"  (testing-extra)  and 
"TXV"  (testing-extra-visual)  specifications  are  preferred  in 
Air  Force  systems.  These  require,  in  addition  to  standard  mil- 
itary quality  tests,  the  performance  of  a burn-in  (operation 
at  full  ratings  for  a period  of  time,  usually  168  hours)  to 
cause  parts  with  latent  defects  to  fail.  Those  parts  are  thus 
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excluded  from  the  batch.  "TXV"  specifications  require  a visual 
inspection  of  the  part  before  the  lid  is  hermetically  sealed  on, 
in  addition  to  the  burn-in. 


Microcircuits  purchased  under  MIL-M-38510,  Microcircuits  General 

Specification,  are  required  to  be  tested  to  a defined  quality  ] 

level  using  test  methods  of  MIL-STD-883,  Test  Methods  and  Pro-  ] 

cedures  for  Microelectronics.  Class  S represents  the  highest  j 

level  for  use  in  critical  systems.  Class  B represents  Air  Force  * 

preferred  quality  for  normal  usage,  and  Class  C a relatively  low  i 

quality  for  those  rare  cases  when  reliability  is  not  a great  con-  \ 

cern,  such  as  an  extremely  simple  item  in  a non-critical  applica-  I 

tion.  However,  even  Class  C parts  have  failure  rates  several  ; i 

times  better  than  commercial  products  (see  MIL-HDBK-217) . j 1 

Selection  of  parts  is  therefore  an  extremely  important  matter, 
and  in  full  scale  development  is  critical.  During  the  valida- 
tion phase,  the  use  of  high  grade  parts  is  not  always  essential, 
if  provision  is  made  for  their  use  in  later  stages.  Lower  grade 
parts  may  often  be  used  in  validation  units  not  scheduled  for  re- 
liability testing,  but  their  form-fit-function  must  be  the  same 
as  the  higher  grade  parts  to  be  used  later  in  the  full  scale  de- 
velopment and  production  units,  permitting  direct  substitution 
without  design  changes.  However,  the  selection  of  parts  to  meet 
the  reliability  requirements  must  begin  in  the  first  equipments 
built.  Preparation  of  preferred  parts  lists  can  be  fruitful  even 
i in  the  conceptual  phase  where  at  least  part  policies  must  be  es- 

tablished. 


Only  microcircuits  listed  in  MIL-STD-1562  and  procured  in  accord- 
ance with  MIL-M-38510  are  standard  for  new  design.  When  non- 
stanoard  microcircuit  devices  are  approved  for  use,  the  general 
requirements  of  MIL-M-38510  apply.  Nonstandard  devices  must  be 
screened  and  qualified  in  accordance  with  the  requirements  of 
MIL-STD-883.  Only  JANTX  semiconductor  devices  selected  from 
MIL-STD-701  are  standard.  When  a JANTX  device  is  not  listed, 
the  selection  of  nonstandard  devices  must  conform  to  the  follow- 
ing order  of  precedence:  (a)  a JAN  device  listed  in  MIL-STD-701, 
(b)  a JAN  device  covered  by  MIL-S-19500  but  not  listed  in  MIL- 
STD-701,  and  (c)  a commercial  device.  As  a minimum,  a TX  burn-in 
should  be  required  for  all  nonstandard  devices. 

Review  of  parts  lists  is  not  only  necessary  to  assure  the  use  of 
preferred  quality  levels,  but  also  to  assure  that  currently 
preferred  versions  are  used  and  that  future  availability  and 
cost  are  considered.  Also,  nonstandard  parts  are  often  neces- 
sary and  the  review  procedure  must  make  sure  that  adequate 
qualification  and  reliability  screening  tests  are  applied  to 
those  parts. 
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Parts  selection  and  control  are  so  important  that  review 
board  procedures  are  used  to  help  assure  the  Air  Force  that 
the  job  is  done  with  its  long-  rm  interests  protected. 

These  interests  include  not  only  reliability,  but  also  main- 
tainability, logistics  supportabil ity,  commonality,  availa- 
bility, and  cost  of  parts.  All  have  direct  impact  on  system 
life  cycle  costs.  MIL-STD-965  Parts  Control  Program,  defines 
criteria  and  guidelines  for  setting  up  parts  control  procedures 
in  a contract.  The  procedure  selected  is  at  the  discretion  of 
the  procuring  activity.  Basically,  two  types  of  control  programs 
are  outlined.  One  is  for  large  system  procurement  employing  a 
prime  system  integration  contractor,  and  requires  the  use  of  a 
Parts  Control  Board  (PCB).  This  is  procedure  II  of  MIL-STO-965. 
Procedure  I,  a less  formal  setup  is  generally  used  for  small  equip- 
ment development  programs.  Both  procedures  require  procuring 
agency  approval  of  parts  selected  for  use  in  the  hardware. 

The  Chairman  for  the  PCB  is  usually  the  prime  contractor.  However, 
someone  else  may  be  designated  as  Chairman  of  the  PAG  by  the 
procuring  activity.  While  the  procuring  activity  always  has  the 
right  to  disapprove  PCB  actions,  the  PCB  normally  makes  the  part 
selection  decisions.  This  organizational  arrangement  requires 
Air  Force  management  emphasis  and  participation,  since  board 
chairmanship  is  often  in  the  hands  of  a contractor  whose  long- 
term interests  and  motivation  may  not  correspond  with  that  of 
the  Air  Force.  Short-term  goals  are  often  an  overriding  concern 
for  the  contractor. 

To  aid  the  procuring  activity  in  parts  approval  decisions,  the 
Military  Parts  Control  Advisory  Group  (MPCAG)  may  be  employed. 

This  is  a Department  of  Defense  organization  which  provides 
advice  to  the  military  departments  on  the  selection  of  parts 
in  assigned  commonality  classes.  Primary  contact  points  are 
the  Defense  Electronics  Supply  Center,  Dayton  Ohio,  for 
electrical  and  electronic  parts,  and  the  Defense  Industrial 
Supply  Center,  Philadelphia  PA  for  mechanical  parts.  Support 
for  semiconductors,  printed  circuit  boards  and  circuit  board 
connectors  is  also  available  from  the  Rome  Air  Development 
Center,  Griffiss  AFB,  NY. 

In  summary,  parts  selection  and  control  are  extremely  important 
development  program  activities  which  can  cause  significant  de- 
velopment costs.  Furthermore,  they  can  be  a significant  admin- 
istrative burden  to  the  Air  Force  Project  Office  and  to  support 
Government  laboratories  and  supply  centers.  Because  of  the 
sheer  volume  of  parts  decisions  in  large  programs,  and  the  time 
and  cost  pressures  involved,  it  is  tempting  for  parts  control 
activities  to  cut  corners,  simplify  the  procedures,  or  drift 
towards  a rubber-stamp  type  operation.  These  tendencies  must 
be  resisted  by  the  Air  Force  project  manager,  and  the  Air  Force 
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staff  people  should  carefully  question  the  adequacy  of  sup- 
port. The  Impact  on  life  cycle  cost,  system  effectiveness, 
and  logistic  supportablllty  Is  great.  The  cost  of  a good 
parts  program  can  be  considerable  but  the  costs  of  later 
failures,  re-design,  and  retesting  can  make  It  Inexpensive 
by  comparison. 


and  a significant  administrative  burden. 


DESIGN  REVIEW  (MIL-STD-1521 ) 

The  design  review  has  both  management  and  engineering 
aspects.  To  the  manager.  It  is  a controlling  activity.  To 
the  engineer.  It  Is  a technical  critique  of  the  work  accom- 
plished. Therefore,  the  design  review  is  an  activity  where 
management  and  engineering  are  closely  coupled.  It  Is  a 
powerful  management  tool  for  the  Air  Force  program  manager, 
and  he  must  assume  a personal  role  In  Its  conduct. 

During  the  design  review,  all  reliability  efforts  leading  to 
design  decisions  are  formally  reviewed.  This  Includes  require- 
ments, modeling,  predictions,  apportionment,  failure  modes  and 
effects  analysis,  parts  selection,  and  overall  design  for  re- 
liability. Because  of  the  volume  of  these  tasks,  the  review 
Is,  of  course,  done  at  a summary  level  of  detail.  In  an  ef- 
ficiently run  program,  a series  of  Informal  detail  design 
reviews  will  be  conducted  between  designer  and  supporting 
engineers  within  the  contractor's  facility.  In  addition, 
the  contractor  should  seek  consultation  with  Air  Force 
specialists  to  Iron  out  any  questions  In  advance.  Then 
the  formal  design  review  should  run  smoothly  and  be  a 
summary  of  key  decisions  for  the  Air  Force,  along  with 
supporting  rationale. 
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Another  comment  should  be  made.  In  an  effectively  managed 
program,  the  designer  will  consider  reliability  engineers 
and  other  supporting  agencies  as  partners  in  meeting  program 
goals,  rather  than  as  critics  to  be  placated.  This  attitude 
is  fostered  only  when  management  treats  reliability  as  an  es- 
sential design  parameter,  rather  than  a necessary  evil  which 
interferes  with  the  designer's  flexibility. 

Design  reviews  are  a normal  part  of  a development  program.  In- 
formal design  reviews,  where  the  most  significant  effort  should 
be  made,  are  the  responsibility  of  the  system  contractor.  For- 
mal reviews  require  Air  Force  participation,  and  serve  as  con- 
trol activities  which  assure  that  reliability,  among  other  things, 
has  been  built  Into  the  design. 


Design  review  is  a very  important  critique  and  control  activity. 


^DEVELOPMENT  SPECIFICATIONS 

The  Government's  ultimate  control  over  the  hardware  pro- 
ducer Is  through  contractual  specifications.  These  specifi- 
cations are  Incorporated  Into  the  statement  of  work  which  be- 
comes part  of  the  contract.  The  statement  of  work  not  only 
includes  overall  equipment  or  system  performance  requirements, 
but  also  specific  design  restrictions  necessary  for  military 
systems.  These  restrictions  generally  support  the  goals  of 


standardization,  logistic  supportability,  reliability,  main- 
tainability, safety,  configuration  control  and  so  on.  In 
total,  they  may  be  costly  restrictions  during  development,  but 
money-saving  over  the  long  run. 

Selection  of  appropriate  specifications  is  initially  an  Air 
Force  task  under  the  direction  of  the  project  manager.  These 
specifications  go  into  the  first  contract  of  the  conceptual 
phase.  As  the  program  proceeds,  more  detailed  specifications 
will  evolve  from  contractor  design  effort  and  these  will  be 
inserted  into  later  contracts,  but  only  after  careful  review 
by  the  project  manager's  engineering  staff.  The  development 
of  specifications  is  a continuing  task  for  both  the  Air  Force 
and  the  contractor  until  a production  contract  is  solicited. 

The  preparation  of  specifications  by  a contractor  is  expensive. 
Therefore,  the  Air  Force  should  take  advantage  of  any  suitable 
standard  military  component  specifications  which  exist.  For 
example,  military  specifications  now  exist  for  over  500  standard 
microcircuit  devices  under  MIL-M-38510,  Microcircuit  General 
Specification,  and  new  ones  are  being  added  continuously.  It 
would  be  irresponsible  for  the  project  manager  to  allow  a con- 
tractor to  repeat  any  of  this  work  which  has  already  been  care- 
fully done  by  Government  laboratories.  Furthermore,  these  Gov- 
ernment specifications  insure  uniform  and  prediptable  quality, 
standardization,  and  lower  life  cycle  costs.  They  also  save 
development  dollars  which  can  be  better  spent  on  unique  system 
design  tasks  which  the  Government  laboratories  are  not  able  to 
handle. 


Many  system  design  requirements  and  considerations  which  affect 
reliability  are  contained  in  the  standard  references  (boiler 
plate)  found  in  almost  every  contract.  For  example,  MIL-E-5400, 
General  Specification  for  Airborne  Equipment,  is  included  in  al- 
most every  avionic  system  specification.  MIL-E-5400,  in  turn, 
references  MIL-STD-454,  General  Requirements  for  Airborne  Equip- 
ment, which  in  turn  references  preferred  parts  specifications  and 
other  design  requirements  necessary  for  production  of  reliable 
electronic  equipment.  This  "boiler  plate"  represents  years  of 
experience  and  is  a practical  response  to  the  familiar  quotation, 
"They  who  do  not  learn  from  the  past  are  condemned  to  repeat  it". 
Repetition  of  past  mistakes,  such  as  the  use  of  dissimilar  metals 
causing  galvanic  corrosion,  is  encountered  in  the  best  of  efforts. 
Yet  contractors  should  not  be  discouraged  from  challenging  boiler 
plate  requirements  that  they  consider  unrealistic.  Such 
challenges  should  be  given  a complete  technical  evaluation  by  the 
program  manager  and  his  technical  experts. 

The  boiler  plate  alone,  however,  is  not  enough.  First  of  all, 
quantitative  reliability  requirements  tailored  to  your  program 
must  be  clearly  defined  in  the  development  specification.  This 
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is  never  in  the  boiler  plate.  Furthermore,  reliability  can  be 
defined  in  various  ways  (i.e.  operational  reliability,  serial 
reliability,  etc.)  and  it  must  be  clearly  stated  what  defini- 
tion applies  to  the  quantitative  requirements.  Standard  termi- 
nology for  Air  Force  reporting  of  reliability  is  contained  in 
AFR  80-5,  but  these  definitions  are  not  necessarily  levied  on 
the  contractor.  The  contractor  requirements  must  be  trans- 
latable into  standard  terminology  for  Air  Force  use,  and  must 
be  clearly  understood  by  the  contractor  and  the  procuring 
activity.  Any  special  requirements,  such  as  the  use  of  higher 
grade  parts  than  normal,  special  screening  techniques,  equip- 
ment burn-in,  reliability  evaluation  tests,  etc.  must  be 
specified.  The  desired  elements  of  the  reliability  program 
must  also  be  defined.  This  includes  such  items  as  predictions 
to  be  made  and  the  methods  to  be  used,  failure  modes  and 
effects  analysis,  design  reviews,  data  submittals,  data  report- 
ing systems,  component  failure  analysis,  etc.  Reliability 
standards  such  as  MIL-STD-785,  Reliability  Program  for  System 
and  Equipment  Development  and  Production  must  be  referenced 
in  whole  or  in  part  since  the  degree  of  application  must  be 
tailored  to  the  procurement  by  the  procuring  activity.  The 
use  of  MIL-HDBK-217,  Reliability  Prediction  of  Electronic 
Equipment  should  be  specified,  with  deviations  subject  to  the 
approval  of  the  procuring  activity.  Reliability  acceptance 
testing  is  also  a critical  specification  requirement.  Specific 
test  plans  and  environments  must  be  selected  from  MIL-STD-781, 
Reliability  Design  Qualification  and  Production  Acceptance 
Tests:  Exponential  Distribution,  Another  example  is  MIL-STD- 
965  which  defines  the  procedures  for  parts  control  and  stan- 
dardization. A final  example  is  the  temptation  to  use  plastic 
encapsulated  semiconductor  devices  and  micro-circuits  which 
are  not  included  in  any  standard  part  specification.  However, 
they  might  be  accepted  as  nonstandard  parts  through  a Parts 
Control  Board  decision.  Because  of  the  historically  poor 
reliability  of  these  devices,  it  is  coimion  practice  to  include 
in  the  development  specification  a specific  prohibition  of 
their  use. 

Thus  a great  deal  of  attention  to  reliability  inputs  for  the 
development  specification  is  necessary,  and  the  Air  Force 
program  manager  must  rely  on  his  reliability  specialists. 

Because  of  differences  between  programs,  and  continuous  changes 
in  engineering  technology  and  reliability  methods,  every  pro- 
curement will  require  a unique  set  of  reliability  requirements. 

At  the  system  level,  the  evolution  of  the  development  specifi- 
cation starts  in  the  conceptual  phase  and  continues  through 
validation  into  full  scale  development.  The  specification 
used  in  full  scale  development  must  be  complete  and  unambiguous. 
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Acceptance  Specifications 

A critical  aspect  of  the  procurement  documentation  is  the 
system  or  equipment  acceptance  specification.  Acceptance  cri- 
teria, are,  of  course,  delineated  in  the  same  procurement  doc- 
uments as  the  design  criteria,  but  are  backed  up  by  another  set 
of  military  standards.  MIL-STD-781 , Reliability  Design  Qualifi- 
cation and  Production  Acceptance  Tests:  Exponential  Distribution, 
provides  a variety  of  reliability  demonstration  plans  and  test 
levels.  The  plans  define  statistical  criteria,  and  the  test 
levels  define  the  severity  of  environmental  conditions.  The 
procurement  documentation  must  state  which  test  plan  and^est 
level  will  be  used  for  reliability  quantification  and,  if  appro- 
priate, which  test  plan  and  level  will  be  used  for  production 
verification.  The  number  of  samples  used  must  be  defined.  In 
addition,  the  measurements  to  be  taken  during  the  test  and  the 
rules  for  considering  a failure  as  relevant  or  non-relevant  must 
be  stated.  A relevant  failure  counts  against  the  equipment  being 
developed  while  a non-relevant  failure  does  not.  Therefore,  a 
non-relevant  failure  must  be  very  carefully  defined,  since  this  is 
a shelter  area  where  the  contractor  may  seek  refuge  if  the 
acceptance  tests  yield  many  failures.  A non-relevant  failure  is 
generally  one  which  is  no  fault  of  the  equipment  being  developed, 
such  as  a failure  in  the  monitoring  test  equipment,  or  a failure 
due  to  equipment  misuse.  Also,  and  extremely  important,  the 
test  environment  must  be  defined. 

Reliability  Evaluation  Tests  and  Reliability  Growth 

A special  reliability  test,  and  a most  often  neglected  tool, 
is  the  reliability  evaluation  test.  This  is  a test  without 
acceptance  criteria,  performed  by  the  contractor  to  obtain  infor- 
mation on  reliability  deficiencies  of  the  hardware. 

When  equipment  is  first  fabricated,  it  ideally  should  meet  its 
reliability  requirements.  In  practice  this  seldom  occurs,  espe- 
cially in  complex  non-digital  systems.  Design  defects,  workman- 
ship problems,  and  parts  defects  all  detract  from  the  inherent 
reliability  potential  of  the  hardware.  These  deficiencies  must 
be  identified,  their  causes  determined,  and  corrective  action 
taken  before  the  hardware  can  demonstrate  its  potential  relia- 
bility. The  "fly  before  you  buy"  concept  was  designed  to  provide 
the  opportunity  for  identification  and  correction  of  hardware  de- 
ficiencies, and  hence  reliability  growth  should  be  an  essential 
feature  of  the  system  validation  phase.  A military  handbook  on 
reliability  growth  is  in  preparation  and  scheduled  for  publication 
in  the  near  future. 
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During  the  conceptual  phase,  it  is  desirable  to  conduct  relia- 
bility evaluation  tests  on  critical  components  being  considered 
for  use  in  validation  models  of  the  equipment  or  system. 

The  magnitude  of  the  reliability  evaluation  and  improvement  ef- 
fort can  be  quite  large.  One  major  electronic  system  development 
contractor  has  issued  a reliability  planning  and  management  guide 
for  its  people  which  claims  that  the  first  model  of  a large  elec- 
tro-mechanical system  will  initially  demonstrate  only  one-tenth 
of  its  inherent  reliability.  Furthermore,  the  guide  says  that 
about  100  times  the  predicted  MTBF  of  test  experience  is  needed 
to  find  and  eliminate  the  reliability  deficiencies.  It  also  says 
that  the  problems  found  are  about  equally  divided  between  parts, 
workmanship,  and  design  deficiencies.  While  the  exact  growth  time 
may  be  questioned,  and  such  efforts  as  parts  screening  can  elim- 
inate many  problems  before  fabrication,  it  is  certain  that  any 
contractor  must  plan  for  a reliability  growth  effort.  The  con- 
tractor's Reliability  Program  Plan  must  acknowledge  the  need  for 
reliability  tests,  failure  reporting,  failure  analysis,  and  cor- 
rective action.  Cost  and  schedule  impacts  can  be  reduced  by  uti- 
lizing other  scheduled  tests  to  provide  reliability  information. 

Any  operational  test  of  the  system  can  be  used  to  obtain  reliabil- 
ity information.  If  sufficient  testing  is  planned  for  other  pur- 
poses, it  is  even  possible  to  eliminate  special  re’Mbility  eval- 
uation tests  entirely.  Of  course,  provision  for  rt. lability  re- 
porting, failure  analysis,  and  corrective  action  must  be  included 
in  the  test  planning  in  any  case. 

The  cost  of  reliability  evaluation  testing  can  be  very  significant, 
depending  upon  the  nature  of  the  equipment  and  the  contractor's  ap- 
proach to  the  problem.  The  effort  is  very  important  during  valida- 
tion and  full  scale  development  and  cannot  be  deleted. 


FAILURE  ANALYSIS 

Failure  analysis 
is  performed  by  the 
hardware  manufacturer. 

It  consists  of  statis- 
tical analysis  of  fail- 
ures to  determine  their 
relative  importance  and 
their  history  of  im- 
provement as  problems 
are  eliminated.  It  also 
consists  of  engineering 
analysis  to  determine 
the  cause  and  cure  of 
each  failure.  The  latter 
includes  the  "autopsy"  of 
failed  parts  to  establish 
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the  physical  or  chemical  cause  of  the  failure.  Such  autopsies 
require  specialized  equipment  such  as  X-ray  machines,  electron 
microscopes,  metal lographs,  hermeticity  test  equipment,  and  so 
on.  While  some  system  contractors  maintain  quite  sophisticated 
failure  analysis  laboratories,  others  depend  on  outside  labora- 
tories or  part  vendors  for  failure  analysis.  The  part  vendor 
is  often  an  unsatisfactory  source  of  failure  analysis  informa- 
tion, and  usually  an  alternative  is  preferable. 

Failure  analysis  is  useful  throughout  a program,  but  the  bulk 
of  the  activity  should  take  place  during  validation  and  initial 
full  scale  development  when  most  reliability  growth  should  occur. 
In  the  conceptual  phase,  failure  analysis  is  desirable  for  criti- 
cal components  being  considered  for  use  in  validation  phase  equip- 
ment. In  production  and  deployment,  failure  analysis  will  be  used 
to  correct  deficiencies  which  jeopardize  the  achieved  reliability. 

The  cost  of  this  effort  is  difficult  to  predict  since  it  depends 
upon  the  number  and  types  of  failures  encountered.  Nevertheless, 
it  must  be  estimated  by  reliability  specialists  and  included  in 
planning. 

DATA  SYSTEM  (AFSCP/AFLCP  400-11) 

Reliability  growth  requires  a data  system  to  assure  the  re- 
porting of  failures  and  implementation  of  corrective  action. 

The  data  system  not  only  documents  failures,  but  also  records 
the  results  of  failure  analysis  discussed  above,  and  the  result 
of  corrective  action  taken.  It  is  the  documentary  communication 
system  upon  which  the  testing-analysis-fixing  cycle  depends.  It 
also  gives  both  the  contractor  and  Air  Force  project  managers 
performance  indicators  which  allow  them  to  measure  progress. 
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The  data  system  must  meet  two  basic  requirements: 

* It  must  record  data  on  failures,  analyses  and 
corrections 

* It  must  be  compatible  with  standard  Air  Force 
data  systems 

In  the  conceptual,  validation,  full  scale  development  and  pro- 
duction phases,  the  data  system  will  be  internal  to  the  con- 
tractors, with  Air  Force  visibility  and  control  through  report- 
ing of  summary  data.  In  the  deployment  phase,  the  Air  Force 
Maintenance  Data  Collection  System  (APR  66-14,  AFM  66-1)  will 
be  used.  AFM  66-1  data  provides  the  input  to  the  DO- 56  Product 
Performance  System.  Therefore,  it  is  important  that  the  contractor's 
data  system  be  compatible  with  the  AFM  66-1  system,  so  that 
valid  comparisons  can  be  made  between  reliability  data  collected 
during  development  and  production,  and  reliability  data  col- 
lected during  operational  use. 

For  development,  test  and  evaluation  programs  conducted  in- 
house  by  AFSC  organizations,  the  AFSC  Systems  Effectiveness 
Data  System  (SEOS)  is  being  promoted  by  AFSC,  and  is  sometimes 
required  (see  AFSC  Supplement  1 to  AFR  80-5).  AFSC  also  re- 
quires that  SEDS  be  used  by  contractors  if  they  do  not  already 
have  an  adequate  data  system.  The  SEDS  data  system  is  designed 
for  use  with  the  CDC  6500/6600  computers  and  is  fully  implemented 
at  Air  Force  Flight  Test  Center,  Edwards  Air  Force  Base, 

California. 

In  large  development  programs,  data  systems  can  produce  data 
quantities  which  are  overwhelming  to  the  project  manager.  He 
must  therefore  be  sure  that  summary  data  is  compiled  in  some 
concise  way  so  that  he  can  quickly  gauge  progress  during  pro- 
gram reviews.  Various  graphical  schemes  are  fairly  easy  to 
devise.  One  contractor  employed  the  simple  technique  of  re- 
quiring a monthly  internal  report  to  the  program  manager  on 
the  ten  most  significant  failures,  including  status  of  corrective 
action.  With  this  kind  of  attention,  there  is  little  doubt  that 
corrective  action  will  get  proper  emphasis.  Whether  the  report 
covers  ten  failures,  twenty  failures,  or  is  weekly  rather  than 
monthly  depends  on  the  size  of  the  program.  Some  programs  may  be 
too  large  to  handle  this  way,  and  a more  statistical  or  graphical 
approach  would  be  best.  Whatever  the  summary  technique  may  be. 

Air  Force  management  can  employ  the  same  method  by  requiring  it 
to  be  presented  during  Air  Force  program  reviews. 
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QUALITY  CONTROL 

Without  an  effective  quality  control  program,  most  of  the 
other  reliability  efforts  would  be  wasted.  For  instance, 
equipment  expertly  designed  for  high  reliability,  including 
the  specificaition  of  the  best  parts  and  materials,  will  fail 
reliability  tests  if  the  equipment  is  manufactured  with  shoddy 
workmanship  and  incoming  parts  and  materials  are  not  inspected 
to  make  sure  they  meet  specifications. 

Reliability  defects  or  problems  can  be  generally  classified 
into  one  of  the  following  four  categories: 

* Equipment  design 

* Parts  and  materials 

* Documentation 

* Workmanship 

The  reliability  program  elements  discussed  thus  far  have  emphasized 
the  first  three  categories.  The  quality  control  program  covers  all 
four  categories  with  emphasis  on  the  fourth,  workmanship.  In  the 
context  of  quality  control,  workmanship  includes  a wide  variety  of 
actions  by  people,  all  reflecting  how  well  the  standards  of  quality 
are  actually  carried  out.  These  actions  include  manufacturing  op- 
erations, purchasing  practices,  testing  procedures,  handling,  stor- 
age, delivery,  and  installation.  The  quality  control  program  must 
insure  that  workmanship  does  not  detract  from  the  inherent  relia- 
bility engineered  into  an  equipment  or  system. 

Essentially  all  contracts  for  equipments  or  systems  in  validation, 
full  scale  development  or  production  will  require  the  use  of  MIL- 
Q-9858,  Quality  Program  Requirements.  This  specification  requires 
the  contractor  to  have  a quality  program  based  on  standards,  rec- 
ords, and  corrective  action.  The  exact  program  is  up  to  the  con- 
tractor, but  it  must  meet  certain  minimum  requirements  and  must  be 
completely  visible  to  Air  Force  quality  control  people.  The  qual- 
ity control  program  must  also  extend  to  subcontractors  and  vendors. 

In  the  deployment  phase,  a quality  control  program  is  also  neces- 
sary to  insure  that  maintenance  procedures,  replacement  parts  and 
equipment  modifications  do  not  detract  from  the  inherent  quality 
of  the  operational  equipment. 
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ENVIRONMENTAL  TESTS 


Before  system  or  equip- 
ment reliability  accept- 
ance tests  begin,  com- 
ponents, subassemblies, 
and  equipments  must  go 
through  environmental 
qualification  tests. 

These  include  such  tests 
as  shock,  vibration,  ac- 
celeration, temperature, 
humidity,  sand,  dust, 
salt  spray,  nuclear  rad- 
iation, electromagnetic 
interference,  and  so  on. 

Acceptance  specifications 
must  include  the  most 
severe  field  environments 
expected.  These  relative- 
ly short  duration  tests 
will  bring  out  failure 
mechanisms  which  may  never 
show  up  in  the  long-term  reliability  acceptance  tests,  even  though 
the  latter  may  include  vibration  and  temperature  cycling.  Therefore, 
these  environmental  tests  are  an  Essential  and  sometimes  costly  part 
of  the  overall  reliability  program. 

The  importance  of  thorough  environmental  tests  is  illustrated  in  the 
results  of  a 1971  study  by  Air  Force  Flight  Dynamics  Laboratory  which 
found  that  52%  of  the  failures  in  the  operational  equipments  studied 
were  environmentally  induced,  reflecting  the  inadequacy  of  environ- 
mental testing  during  development  and  production.  The  study  also 
showed  an  almost  one-to-one  correspondence  between  waiver  of  environ- 
mental requirements  or  tests  and  subsequent  severe  environmental  prob- 
lems in  the  field. 

Individual  environmental  tests  are  relatively  short  in  duration,  but 
a series  of  different  tests  and  retests  is  time  and  equipment  consum- 
ing. Therefore,  they  are  expensive  and  a common  source  of  program 
program  delays.  The  delays  are  usually  due  to  poor  planning  (not 
leaving  a reasonable  time  for  failure  analysis,  design  correction  and 
retest)  or  inadequate  design  or  quality  control  leading  to  excessive 
failures. 

Regardless  of  time  or  cost,  environmental  testing  is  an  essential  part 
of  any  reliability  program  and  not  a safe  place  to  look  for  "money- 
saving" shortcuts.  The  life  cycle  cost  penalty  can  dwarf  any  develop- 
ment cost  savings. 
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Towards  the  end  of  the  validation  phase,  environmental  tests 
are  very  important,  but  not  critical.  Failures  in  this  phase 
need  analysis  along  with  definition  of  corrective  action,  but 
implementation  and  retest  are  not  always  necessary.  It  depends 
upon  the  seriousness  of  the  problem. 

Environmental  tests  are  rated  critical  towards  the  end  of  full 
scale  development.  This  is  when  the  equipment  design  should 
display  full  acceptability  for  production.  Environmental  tests 
must  be  completed  prior  to  the  beginning  of  production.  During 
production,  limited  environmental  tests  are  needed  at  least  per- 
iodically to  insure  that  production  quality  remains  satisfactory 
and  that  production  fabrication  methods  have  not  degraded  the 
capability  of  the  equipment. 


RELIABILITY  ACCEPTANCE  TESTS 


I know  you're  tough, 
but  one  of  f 

these  days.../ 


Reliability  acceptance  tests 
are  designed  to  estimate  the 
MTBF  of  the  equipment.  They  ^ 
may  include  certain  tempera- 
ture cycling  and  vibration 
routines.  A variety  of 
standard  test  procedures  can 
be  selected  from  MIL-STD-781, 
Reliability  Tests:  Exponen- 
tial Distribution,  depending 
upon  the  use  of  the  equipment. 


A clearly  defined  and  closely 
monitored  reliability  accept- 
ance test  is  a critical  pro- 
gram element  at  the  end  of 
full  scale  development.  With- 
out it,  the  contractor  is  not 
likely  to  be  motivated  to  put 
needed  effort  into  other  re- 
liability program  elements  : 

described  above,  except  pos- 
sibly for  the  environmental  tests.  Effort  may  be  diverted  by  the 
contractor  to  those  tasks  which  lead  to  the  timely  delivery  of 
other  data  and  hardware  called  out  by  the  contract.  The  reliability 
acceptance  test  provides  data  to  demonstrate  delivery  of  specified 
reliability  to  the  Air  Force  as  called  out  in  the  acceptance  speci- 
fications. Without  this  data,  the  Air  Force  will  be  simply  banking 
on  faith  and  hope.  A decision  to  go  into  production  cannot  be  made 


prior  to  successful  completion  of  this  test.  A reliability 
acceptance  test  will  not  guarantee  the  achievement  of  spec- 
ified reliability,  but  the  lack  of  an  acceptance  test  will 
almost  certainly  guarantee  that  adequate  reliability  will 
not  be  achieved. 

These  tests  are  rated  very  important  at  the  end  of  valida- 
tion, simply  because  they  are  needed  to  evaluate  the  effec- 
tiveness of  all  the  other  reliability  program  efforts  during 
the  validation  phase.  They  are  critical  at  the  end  of  full 
scale  development,  because  a production  decision  is  due  at 
that  time.  They  need  to  be  repeated  at  the  beginning  of  pro- 
duction, and  again  at  intervals  during  production,  to  insure 
that  production  methods  have  not  degraded  the  inherent  relia- 
bility designed  into  the  equipment. 

Reliability  acceptance  tests  can  be  quite  costly.  Most  of 
the  cost  is  due  to  the  time  required.  Reliability  is  a time- 
dependent  parameter,  and  there  is  no  way  to  verify  reliability 
without  accruing  an  amount  of  test  time  commensurate  with  the 
confidence  required  in  the  test.  All  reliability  tests  are 
statistical  which  means  that  there  is  always  some  risk  of  poor 
equipment  appearing  acceptable  or  good  equipment  appearing  un- 
acceptable. The  longer  the  test  time  the  lower  these  risks 
become.  Hence,  selection  of  a reliability  test  is  always  a 
trade-off  between  risk  and  test  time.  This  trade-off  is  the 
program  manager's  prerogative,  but  he  must  fully  understand 
the  risks  and  cost  of  various  test  alternatives.  Here  again 
the  recommendations  of  a reliability  specialist  are  needed 
to  define  the  most  appropriate  test  routine  to  put  in  the  con- 
tract. 

In  order  to  save  test  costs,  as  well  as  to  enhance  the  validity 
of  the  reliability  tests,  studies  are  underway  to  integrate 
certain  environmental  and  reliability  test  routines  to  a broader 
extent  than  now  reflected  in  MIL-STD-731.  If  these  studies  yield 
definitive  results,  improved  procedures  will  be  incorporated  into 
the  testing  standards. 

SCHEDULING  RELIABILITY  TESTS 

Reliability  testing  takes  time  and  it  is  obvious  that  enough 
time  must  be  provided  in  the  schedule  to  run  the  tests.  Scheduling 
the  reliability  acceptance  test  is  the  greatest  problem.  This  test 
must  be  scheduled  for  late  in  full  scale  development  because  if  it 
were  done  earlier  changes  in  configuration  would  make  it  invalid. 
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On  the  other  hand,  it  cannot  be  scheduled  too  late  for  two 
reasons.  First,  it  is  not  uncommon  for  equipment  to  fail 
the  test.  Therefore,  time  must  be  allowed  for  modification 
of  the  equipment  and  retest  prior  to  the  end  of  the  contract. 
Second,  enough  reliability  testing  must  be  completed  to  make 
a production  decision  which  usually  occurs  before  che  end  of 
full  scale  development.  Most  contracts  require  successful 
completion  of  both  environmental  and  reliability  tests  prior 
to  the  equipment  being  considered  qualified. 

RESOURCE  REQUIREMENTS 

Now  that  the  scope,  con- 
tent, and  purpose  of  an  av- 
erage reliability  program 
are  understood,  the  manager 
needs  to  know  something  a- 
bout  the  engineering  re- 
sources available  to  get 
such  a program  properly 
accomplished.  We  have  em- 
phasized several  times 
that  the  program  has  many 
options  and  must  be  tailored 
to  the  specific  equipment 
or  system  being  developed. 

This  tailoring  is  beyond 
the  capability  of  most  pro- 
gram managers  and  they  need 
professional  help  to  get  the 
job  done  right.  Of  course, 

95%  of  the  technical  work 
will  be  performed  by  the 
equipment  contractor,  but 
the  other  5%  which  is  per- 
formed by  the  Air  Force  (requirements,  specifications,  reviews, 
test  monitoring,  etc.)  is  crucial  to  success.  The  brief  obser- 
vations and  suggestions  below  focus  on  this  crucial  5%. 

TECHNICAL  MANPOWER 

The  Air  Force  program  manager  needs  the  services  of  a relia- 
bility engineer.  Large  program  offices  such  as  the  F-15 
will  usually  have  trained  specialists  assigned  full  time  to  the 
program.  Other  activities  will  depend  on  outside  help.  This  out- 
side help  can  come  from  several  sources.  For  example,  each  of  the 
AFSC  product  divisions  has  a reliability  staff  office  which  can 


provide  part  time  or  temporary  assistance  to  specific  pro- 
grams in  the  division.  The  Reliability  Branch  at  the  Rome 
Air  Development  Center  is  a focal  point  of  reliability  know- 
how in  the  electronics  field,  and  is  frequently  asked  to  pro- 
vide reliability  engineering  support  to  the  AFSC  product  di- 
visions and  laboratories.  Finally,  contractual  support  is 
available.  Space  and  Missile  Systems  Organization  employs 
large  numbers  of  contractor  personnel  supplied  by  TRW  and 
Aerospace  Corporation  for  reliability  support,  and  the  Elec- 
tronic Systems  Division  obtains  some  reliability  support  from 
MITRE  Corporation.  Private  engineering  organizations  not  en- 
gaged in  hardware  production  can  be  hired  also.  For  example, 
ARINC  Research  Corporation,  Battel le  Laboratories,  and  the 
Illinois  Institute  of  Technology  Research  Institute  have  sup- 
ported Air  Force  programs.  Finally,  in  some  situations,  hard- 
ware manufacturers  can  be  employed.  Such  situations  might  in- 
clude reliability  improvement  programs  for  equipments  in  the 
Air  Force  operational  inventory.  Of  course,  hardware  manu- 
facturers should  not  monitor  the  reliability  efforts  of  com- 
petitors. Hence,  though  it  may  sometimes  require  a service 
contract,  there  are  many  avenues  open  to  the  manager  who  needs 
reliability  engineering  support. 

When  time  permits,  the  program  manager  can  also  provide  his  per- 
sonnel with  reliability  training  through  courses  at  the  Air  Force 
Institute  of  Technology  and  several  other  educations  programs 
listed  in  Appendix  A. 

REFERENCE  PUBLICATIONS 

Commercial  books  on  reliability  engineering  are  plentiful. 
Commercial  books  on  rel  iabi  1 i t$?i^ogram  management  are  far  fewer 
and  concentrate  on  the  manufacturer's  in-house  situation.  The 
Air  Force  R&M  manager,  or  the  contractor  R&M  manager  working  on 
Air  Force  programs,  should  have  a library  which  contains  the  fol- 
lowing documents  (in  addition  to  this  management  guide); 

* AFR  80-5,  Air  Force  Reliability  and  Maintainability 
Program,  and  AFSC  Supplement  1. 

These  documents  provide  the  manager  with  the  Air  Force  policy 
on  reliability  requirements  necessary  for  various  types  of  con- 
tracts. Incidentally,  a programmed  guide  to  the  policy  of  AFR 
80-5  is  contained  in  the  Proceedings  of  the  1972  Annual  Relia- 
bility and  Maintainability  Symposium  published  by  the  Institute 
of  Electrical  and  Electronics  Engineers  (IEEE).  This  guide  uti- 
lizes logical  flow  diagrams  to  simplify  the  selection  of  R&M  re- 
quirements appropriate  for  a particular  kind  of  development 
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program.  It  should  be  noted  that,  while  these  documents  pro- 
vide such  guidance  as  whether  or  not  a reliability  test  is  re- 
quired for  a particular  program,  they  do  not  provide  guidance 
In  determining  such  technical  details  as  test  length,  test  con- 
ditions, etc.  Such  details,  as  well  as  the  quantitative  design 
requirements,  the  specific  reliability  tasks  to  be  performed, 
and  the  data  Items  to  be  delivered,  must  be  determined  by  the 
program  manager  together  with  his  supporting  reliability  spe- 
cialist. 


* MIL-STD-785,  Reliability  Program  for  Systems  and 
Equipment  Development  and  Production.  This  document  describes 
the  various  tasks  making  up  a reliability  program.  For  many 
systems  In  full-scale  development.  It  may  be  applied  In  total 
as  a requirement  for  the  contractor.  However,  the  document 
will  not  provide  quantitative  requirements  or  cite  a specific 
demonstration  plan.  It  must  be  reviewed  to  determine  the  ex- 
tent to  which  the  tasks  are  applicable  to  your  program.  In 
many  programs,  only  certain  tasks  should  be  used,  and  your 
statement  of  work  must  specify  which  ones. 

* MIL-STD-781,  Reliability  Design  Qualification  and 
Production  Acceptance  Tests:  Exponential  Distribution.  This 
document  describes  the  various  test  plans  and  environmental 
test  levels  that  may  be  used  for  reliability  demonstration. 

From  these  plans,  the  R&M  manager  must  select  the  best  for 
his  program,  and  specify  It  In  the  statement  of  work.  The 
selection  must  be  a satisfactory  trade-off  between  the  risks 
Involved,  the  test  time,  and  the  number  of  test  samples  which 
can  be  purchased.  These  test  plans  are  based  on  an  exponential 
distribution  of  failures,  which  means  that  the  failure  rate  of 
the  system  Is  essentially  constant.  This  Is  a valid  assumption 
for  electronic  systems.  However,  Items  such  as  engines  which 
exhibit  a predominantly  time  dependent  failure  rate  due  to  wear- 
out  effects,  cannot  validly  etnploy  test  plans  from  MIL-STD-781. 
For  such  items,  a reliability  test  based  on  a defined  minimum 
life  must  be  designed.  This  Is  a fairly  simple  job  for  any  com- 
petent statistician. 

* MIL-HDBK-21 7 , Reliability  Prediction  of  Electronic 
Equipment.  This  document  provides  failure  rates  for  electronic 
and  some  electromechanical  parts  as  a function  of  the  stress 
applied  to  them.  It  also  contains  Instructional  Information 

on  reliability  prediction  for  assemblies  of  parts.  In 
addition,  the  RADC  Nonelectronic  Reliability  Notebook,  RADC- 
TR-75-22  ((A005657),  has  failure  rates  for  various  nonelectronic 
components  such  as  pumps,  valves,  tanks.  Instruments,  etc.,  found 
in  large  electromechanical  systems.  Nonelectronic  part  failure 
data  is  also  published  in  NRPD-1  "Nonelectronic  Parts  Reliability 
Data,  1978"  published  by  the  Reliability  Analysis  Center,  a POD 
Information  Analysis  Center  located  at  RADC. 
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*AFSCP/AFLCP  400-11,  Reliability  and  Maintainability  Data 
Sources.  This  pamphlet  contains  descriptions  of  some  twenty  sources 
of  reliability  and  maintainability  data.  The  sources  cover  a broad 
range  of  parts,  equipments  and  systems  including  electronics,  propulsion, 
missiles,  auxiliary  power  units,  aircraft  structures,  and  so  on.  It 
is  especially  useful  for  the  Air  Force  and  contractor  reliability  engineer 
who  is  searching  for  failure  rate  data  upon  which  to  base  realistic 
reliability  requirements  or  reliability  predictions. 

The  various  reliability  textbooks,  and  the  proceedings  of  the  Annual 
Reliability  and  Maintainability  Symposium,  can  also  be  consulted  for 
educational  purposes  or  in  search  of  solutions  to  particular  problems. 

In  preparation  of  this  writing  are  two  other  documents  of  interest  to 
the  Air  Force  Reliability  Manager.  The  first  is  "Reliability  Growth 
Management",  a proposed  military  handbook  in  preparation  by  a Tri -Service 
Committee  and  scheduled  for  publication  about  July  1979.  Schedules  for 
the  same  time  is  a RADC  Exhibit  on  Reliability  Testing  Using  Prior  Data. 
This  will  provide  test  plans  which  incorporate  existing  information  (the 
"prior").  The  advantages  of  testing  using  a prior  are  a more  meaningful 
definition  of  test  risks  and  the  potential  for  reduced  test  time,  at  no 
loss  of  confidence,  if  the  prior  is  favorable. 

COMPUTERIZED  PREDICTION 

The  Rome  Air  Development  Center  offers  to  Air  Force  managers  a 
computer  program  for  performing  reliability  predictions.  The  program  is 
resident  in  the  RADC  computer  and  may  be  assessed  through  the  ARPA 
Computer  Network  or  by  direct  lines  to  RADC.  On  arrangement  with  RADC, 
the  use  of  the  program  may  be  offered  to  contractors  as  government 
furnished  property  to  be  used  in  meeting  contractual  reliability 
prediction  requirements.  Using  the  service  provides  a low  cost  means 
for  performing  reliability  predictions  and  permits  automated  reiterations 
for  revision  and  trade-off  analyses.  It  should  reduce  the  costs  of 
predictions  significantly.  Starting  October  1979,  RADC  will  charge 
program  offices  with  computer  use  fees.  It  is  expected,  however,  that 
the  costs  to  the  program  office  will  still  be  less  than  either  manually 
performed  predictions  or  contractor  procured  programs. 

FACILITIES 

Facilities  for  reliability  engineering  include  reliability 
test  chambers,  failure  analysis  facilities  and  parts  screening 
apparatus.  Access  to  a computer,  while  not  really  essential,  is  a great 
aid  in  analytical  studies  such  as  the  prediction  and  statistical  analysis 
of  failure  trends. 

Test  chambers  have  been  in  use  for  a long  time,  and  it  is  a rare 
equipment  contractor  who  does  not  have  a reliability  test  chamber. 
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In  contrast,  failure  analysis  and  parts  screening  facilities  are 
not  widely  available.  A basic  failure  analysis  capability  is  not 
expensive,  yet  is  not  available  in  every  manufacturer's  plant. 
More  sophisticated  facilities,  utilizing  expensive  equipment  such 
as  scanning  electron  microscopes,  are  found  only  in  the  larger 
industrial  plants.  Lacking  failure  analysis  facilities,  the 
contractor  is  dependent  on  outside  laboratory  support  or 
analysis  by  the  part  vendor.  Parts  screening  facilities  are 
necessary  only  when  the  contractor  cannot  obtain  the  desired 
screening  from  his  part  vendors,  or  finds  it  more  economical 
to  do  his  own. 


Air  Force  facilities  for  reliability  engineering  in  the  pro- 
gram office  need  include  only  office  space,  an  appropriate 
reference  library,  and  possibly  access  to  a computer.  On  a 
broader  level,  the  electronic  failure  analysis  and  reliability 
research  and  development  laboratories  at  Rome  Air  Development 
Center  provide  an  Air  Force  in-house  facility  from  which  the 
program  manager  can  seek  technical  support.  Availability,  of 
course,  depends  upon  workload  and  your  program  priority. 

SYNOPSIS 

Hardware  reliability  engineering  is  a thoroughly  developed 
discipline,  especially  in  the  electronics  area,  with  the  standard 
reliability  program  elements  described  in  MIL-STD-785.  The  ex- 
ecution of  these  elements,  however,  must  be  tailored  to  the  ob- 
jectives and  needs  of  the  particular  equipment  being  developed, 
and  are  subject  to  execution  with  varying  degrees  of  emphasis 
and  skill  by  contractor  personnel.  Therefore,  the  program  man- 
ager needs  expert  advice  to  prepare  statements  of  work  and  to 
monitor  the  contractor's  efforts. 


The  program  elements  which  stand  out  as  particularly  critical 
are  the  identification  of  realistic  requirements,  selection  of 
quality  parts  and  materials,  thorough  environmental  and  relia- 
bility evaluation  tests,  and  carefully  planned  and  executed  re- 
liability acceptance  tests. 


Chapter  5 


ELEMENTS  OF  A MAINTAINABILITY  PROGRAM 


INTRODUCTION 

A maintainability  assurance  program  involves  coordinated  perform- 
ance of  a series  of  tasks  beginning  with  the  conceptual  phase  and 
continuing  through  full  scale  development.  Maintainability  is 
largely  determined  by  the  overall  configuration  of  the  system  or 
equipment  and  is  pretty  much  fixed  by  the  end  of  full  scale  de- 
velopment. The  job ’had  better  be  done  right  by  that  time,  because 
retrofit  changes  to  enhance  maintainability  during  production  and 
deployment  are  extremely  expensive  and  disruptive. 

Maintainability  is  an  attribute  directly  linked  to  the  manual 
skills  of  people,  and  therefore  is  directly  related  to  human  engi- 
neering and  human  factors  in  design.  Maintainability  is  also  di- 
rectly linked  to  logistics  planning  for  maintenance  and  support, 
and  continuous  coordination  with  AFLC  throughout  the  development 
program  cannot  be  overemphasized. 

As  one  might  suspect,  maintainability  and  maintenance  have  not 
been  developed  into  a deterministic  engineering  discipline  to  the 
extent  that  reliability  has.  There  is  more  of  a subjective  flavor, 
because  of  the  human  factors  involved.  Nevertheless,  maintaina- 
bility engineering  is  organized  into  an  orderly  sequence  of  steps 
which  can  lead  to  reasonably  predictable  and  measurable  results. 
This  chapter  explains  the  engineering  and  management  elements  which 
will  yield  those  results. 

STANDARD  MAINTAINABILITY  PROGRAM  ELEMENTS 

A hardware  maintainability  program 
has  many  similarities  with  a hard- 
ware reliability  program,  as  the 
following  chart  shows.  The  list 
of  maintainability  tasks  is  based 
on  MIL-STD-470,  with  some  editorial 
changes.  Again,  the  ratings  of  rel- 
ative importance  of  the  tasks  are 
subjective  judgments  from  experience 
and  apply  to  an  "average"  maintain- 
ability program.  As  with  the  relia- 
bility program,  the  emphasis  on  sep- 
arate tasks  must  be  tailored  to  the 
specific  equipment  being  developed 


MIL-STD-470 
Maintainability 
Program  Pvequirements 
(For  Systems  and 
Equipments) 
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MAINTAINABILITY  PROGRAM  ELEMENTS 


Element 


Life  Cycle  Phase 

Conceptual 

Validation 

Full  Scale 
Development 

Production 

Deployment 

Requirements  Definition 
Maintenance  Concept 
Maintainability  Analysis 
Design  for  Maintainability 
Maintainability  Prediction 
Design  Review 
Design  Specifications 
Acceptance  Specifications 
Detailed  Maintenance  Plan 
Data  System 
Technical  Manuals 
Maintainability  Acceptance  Tesi 
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and  this  tailoring  must  be  done  by  a maintainability  engineer 
motivated  to  protect  the  Air  Force's  long  term  interests. 

Maintainability  is  simply  a measure  of  the  speed  with  which 
something  can  be  fixed  or  checked  over.  More  formally,  it  is 
defined  as  a characteristic  of  design  and  installation  expressed 
as  the  probability  that  an  item  will  be  restored  to  (or  retained  in) 
a specified  condition  within  a given  period  of  time  using  certain 
procedures  and  tools.  The  "retained  in"  case  usually  refers  to 
preventive  maintenance.  A typical  maintainability  specification 
might  say,  for  example,  that  90%  of  all  failures  must  be  repaired 
in  less  than  15  minutes  using  certain  test  equipment,  tools,  spare 
parts  and  personnel,  and  also  that  the  mean-corrective-maintenance- 
time or  mean-time- to-repair  (MTTR)  will  be  five  minutes.  With  these 
two  parameters,  a corresponding  maintainability  demonstration  test 
can  then  be  selected  to  determine  whether  the  equipment  meets  those 
requirements . 

The  median  time  to  affect  a repair  (a  time  which  will  be  bettered 
by  50%  of  all  repairs  and  exceeded  by  the  other  50%)  may  also  be 
used  as  a figure  of  merit.  For  preventive  maintenace,  the  frequency 
(e.g.,  the  mean  time  between  scheduled  maintenance  actions)  is 
usually  specified  as  well  as  the  duration.  (Note:  The  mean  time 
between  unscheduled  maintenance  is  equivalent  to  the  mean  time 
between  failures  which  is  a reliability  parameter  rather  than  a 
maintainability  figure  of  merit). 

An  important  measure  of  maintainability  which  is  usually  invoked 
for  avionic  equipment  is  maintenance  man-hours  per  flying  hour. 

While  obviosuly  related  to  the  time  to  effect  a repair,  it  also 
includes  consideration  of  the  number  of  personnel  required  which 
directly  affects  the  support  requirements.  It  is  possible  to 
specify  maintenance  man-hours  per  operating  hour,  which  would 
cover  ground  equipment,  though  this  has  seldom  been  done. 

Finally,  the  skill  levels  of  the  maintenance  personnel  are  an 
important  consideration  and  maintainability  requirements  must 
include  the  skill  levels  involved  (e.g.,  the  equipment  shall 
possess  a mean  time  to  repair  of  30  minutes  when  maintained  by 
personnel  of  skill  level  3,  as  defined  in  AFR  35-1). 
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TESTABILITY 


A recent  development  in  maintainability  engineering  is  the  concept 
of  testability.  This  refers  to  the  design  of  cost  effective  fault 
detection  and  isolation  (FO/I)  capabilities  within  a system. 

Obviously,  FO/I  parameters  impact  maintainability.  One  cannot 
predict  or  demonstrate  maintainability  without  considering  the 
FD/I  methods  used.  Yet,  FD/I  design  has  been  a neglected  discipline. 
The  specification  of  FO/I  parameters  is  not  standardized,  and 
a recent  study  found  35  different  figures  of  merit  used  in  various 
Air  Force  procurements.  Until  recently,  there  has  been  no  method 
for  demonstrating  or  evaluating  FD/I  capability.  As  a result, 
the  FD/I  capabilities  of  Air  Force  systems  have  been  virtually 
uncontrolled,  and  indication  of  poor  FD/I  performance  has  aroused 
high  level  Air  Force  concern. 

In  answer  to  the  need  for  a standardized  testability  discipline, 

RAOC  initiated  a broad  study  program  in  FY-78.  The  results  will 
be  available  in  FY-79  and  will  ultimately  be  incorporated  into  the 
existing  maintainability  standards. 

In  the  following  discussion  of  maintainability  program  elements, 
testability  considerations  will  be  described  and  such  guidance 
as  now  exists  provided. 

DEFINITION  OF  REALISTIC  REQUIREMENTS 

Like  the  reliability  program,  the  identification  of  realistic 
requirements  is  critical.  Initial  requirements  should  be  sought 
from  the  operational  command,  since  they  are  best  able  to  visualize 
the  dynamics  of  field  u$e.  However,  these  initial  requirements  may 
be  stringent,  and  should  be  considered  negotiable  by  everyone 
concerned  in  the  early  phases  of  development.  Too  stringent  a 
requirement  (repair  time  too  short)  will  require  sophisticated  fault 
location  methods  with  attendant  cost,  weight  and  perhaps  reliability 
problems,  and  may  unnecessarily  complicate  design  of  the  package 
for  quick  tear-down  and  assembly.  It  can  also  lead  to  apprdaches 
which  merely  transfer  the  repair  problem  to  a remote  facility  with 
a possible  increase  in  total  repair  time  and  other  support  costs 
through  transportation  and  repair  of  large  modules.  On  the  other 
hand,  too  loose  a requirement  will  increase  equipment  downtime, 
which  will  then  reduce  operational  readiness  which  in  turn  increases 
force  size  requirements  at  great  expense.  Therefore,  the  user, 
supporter  and  developer  must  conduct  an  iterative  examination  of 
the  MTTR  requirements  before  they  are  made  firm. 

These  trade-off  studies  should  employ  systems  effectiveness  analysis 
and  life  cycle  costing  techniques,  where  maintainability  can  be 
parametrically  varied  over  a reasonable  range  to  determine  how  it 
affects  system  effectiveness  and  life  cost.  Of  course,  maintainability 


requirements  must  be  developed  in  harmony  with  reliability  require- 
ments since  they  both  affect  cost  and  effectiveness  in  an  inter- 
dependent way.  That  is,  very  high  reliability  eases  the  maintenance 
problems  since  the  item  need  not  be  fixed  very  often.  Low  reliability, 
on  the  other  hand,  will  call  for  a rapid  fix  capability  to  maintain 
the  same  item  effectiveness. 

Considering  testability,  a stringent  MTTR  requirement  may  dictate 
the  use  of  an  extensive  built-in-test  system  with  attendent  weight 
and  cost  penalties.  These  must  also  be  weighed  against  the  need 
for  rapid  repair.  FD/I  parameters  (percent  of  faults  to  be 
detected  by  FD/I,  allowable  false  alarm  rates,  ambiguity  of  fault 
location,  etc.)  must  also  be  specified  and  must  be  realistically 
achievable  and  demonstrable.  Guidance  will  be  available  from  an 
RADC  study,  "BIT/External  Tester  Figures  of  Merit  and  Demonstration 
Techniques",  scheduled  for  completion  June  1979,  though  the  final 
report  will  probably  not  be  published  until  September  1979. 

There  are  many  trade-offs  involved  in  the  selection  of  maintain- 
ability requirements,  and  these  must  be  made  during  the  conceptual 
and  validation  phases  so  that  firm  requirements  are  available  for 
full-scale  development.  The  requirements  must  be  ultimately  set 
by  the  program  manager,  but  the  supporting  analytical  studies  must 
be  made  by  his  staff  and  the  contractor,  with  major  inputs  from  the 
operational  command  and  Air  Force  Logistics  Command  (AFLC). 

MAINTENANCE  CONCEPT 

The  maintenance  concept  is  a statement  of  the  general  policy, 
ground  rules,  and  overall  approach  to  achieving  the  operational 
requirements.  The  concept  includes  such  things  as:  projected 
availability  of  maintenance  facilities  or  equipment  at  field  base  or 
depot;  the  skill  levels  to  anticipate  for  maintenance  personnel,  the 
feasibility  of  contractor  maintenance  support  and  at  what  locations; 
the  necessity  or  desirability  of  using  standard  test  equipment  in  the 
field  or  at  the  depot;  the  kinds  of  transportation  assumed  to  be 
available  from  the  field  to  base  or  depot;  anticipated  transportation 
hazards;  and  so  on.  Finally,  the  concept  will  describe  the  general 
approach  to  maintenance  envisioned  from  the  operational  requirements, 
the  operational  mission,  and  the  logistics  or  maintenance  ground  rules 
which  have  been  listed,  along  with  the  rationale  for  selecting 
this  approach. 

Initially,  the  maintenance  concept  will  be  prepared  by  the  Logistics 
organization  within  the  Program  Office,  if  there  is  one,  or  by  AFLC 
with  using  command  and  AFSC  inputs.  Later  contract  studies  will 
then  re-evaluate  this  concept  in  the  light  of  equipment  design 
studies,  maintainability  analyses,  and  systems  effectiveness  analyses. 

By  the  middle  of  full  scale  development,  the  concept  should  settle 
down  to  a stable  statement  of  ground  rules  on  which  to  base  detailed 
equipment  design  and  maintenance  plans. 
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The  R&M  program  manager  should  note  that  there  is  no  standard 
data  item  in  the  DoD  Index  of  Data  Items  (Acquisition  Management 
Systems  and  Data  Requirements  Control  List)  which  would  be  suitable 
for  delivery  of  a maintenance  concept  under  contract.  While  the 
first  version  would  be  written  by  the  program  office  staff,  a 
practical  approach  to  getting  subsequent  iterations  through  contract 
effort  is  to  combine  the  Maintenance  Requirements  and  the  Maintenance 
Concept  into  an  introductory  section  of  the  Detailed  Maintenance  Plan 
discussed  below.  This  keeps  all  relevant  information  together  in 
a single  document  making  it  easier  for  everyone  to  review.  A suitable 
alternative  would  be  to  define  two  unique  data  items,  one  for  the 
concept  and  one  for  the  Detailed  Maintenance  Plan. 


The  concept  must,  of  course,  consider  testability  in  that  trade-offs 
between  buil t-in-test  and  external  test  and  between  automatic  and 
manual  systems  must  be  reflected  in  the  maintenance  concept. 


Before  continuing  with  our  discussion  of  the  other  maintainability 
program  element:.,  let's  take  a quick  look  at  the  overall  maintain- 
ability program  cycle  depicted  below.  Very  simply,  the  requirements 
come  from  the  user,  the  initial  concept  comes  from  the  user  and  AFLC, 
the  analysis,  design,  and  test  results  come  from  the  AFSC  development 
contractor,  and  the  Detailed  Maintenance  Plan  evolves  through 
successive  iterations  of  all  five  tasks.  The  AFSC  program  manager  is 
responsible  for  the  plan  and  employs  contractors  to  complete  its 


development.  The  Plan  is  ultimately  used  by  AFLC  to  prepare  an 
Integrated  Logistics  Support  Plan  for  use  during  production  and 
deployment.  A cooperative  team  effort  between  user,  supporter, 
and  developer  is  essential  throughout  this  cycle.  While  this 
development  cycle  is  an  iterative  process,  the  requirements  and 
concept  should  be  firm  at  the  beginning  of  full  scale  develop- 
ment and  all  elements  must  be  solidified  towards  the  end  of  full 
scale  development. 
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MAINTAINABILITY  ANALYSIS 

Maintainability  analysis  includes  several  kinds  of  analytical 
efforts  performed  by  the  development  contractor,  all  for  the  pur- 
pose of  deriving  the  best  approaches  to  detailed  equipment  design 
and  maintenance  procedures.  These  analyses  strive  to  find  the 
design  configuration  and  maintenance  procedures  which  will  satisfy 
operational  requirements  within  the  ground  rules  of  the  general 
maintenance  concept  documented  earlier.  These  recommendations 
should  then  be  negotiated  with  the  using  and  supporting  commands. 

A revision  of  the  requirements  and  overall  concept  would  then  be 
incorporated  into  a revision  of  the  Detailed  Maintenance  Plan 
prepared  by  the  development  contractor. 

Maintenance  and  maintainability  analyses  consider  the  tasks 
which  must  be  done  to  restore  an  assumed  equipment  configuration 
to  operation  following  a failure,  and  also  the  tasks  involved 
in  preventive  maintenance  and  replacement  of  consumables.  The 
analyses  should  provide  the  preferred  modular  configuration  or 
packaging  plan;  identify  special  test  equipment  needs;  assist 
in  locating  test  points  and  built-in  fault  location  aids;  define 
the  maintenance  actions  best  performed  in  the  field,  at  inter- 
mediate locations,  or  at  a depot;  provide  guidelines  for  discard- 
at-failure  or  repair  decisions;  provide  maintainability  parameters 
which  permit  the  computation  of  maintainability  predictions;  and 
finally,  guide  the  preparation  of  technical  manuals  for  system 
maintenance.  Thus,  some  form  of  maintenance  or  maintainability 
analysis  is  done  at  all  levels  of  the  system  and  in  all  phases 
of  the  program  through  full  scale  development.  Maintenance  and 
maintainability  analyses  are  reflected  in  the  system  and  equip- 
ment design,  the  support  equipment  recommendations,  the  technical 
manuals,  and  the  Detailed  Maintenance  Plan. 

Maintenance  and  maintainability  analyses  are  accomplished  by  applying 
subjective  judgements  based  on  practical  experience,  and  also 
various  analytical  procedures.  Three  specific  analytical  procedures 
included  in  these  analyses  are  the  following: 
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OPTIMUM  REPAIR  LEVEL  ANALYSIS 


Optimum  Repair  Level  Analysis  (ORLA),  also  known  as  Level  of 
Repair  Analysis  (LORA)  Is  an  analytical  procedure  for  establishing 
the  least  cost  feasible  repair  or  discard  decision  for  maintenance 
actions  at  each  maintenance  level  and  is  Intended  to  Influence 
the  equipment  design  in  that  direction.  It  considers  such  factors 
as  the  cost  of  repairing  a failure  at  the  operational  site  versus 
the  depot,  the  cost  of  discarding  a failed  module  versus  repair, 
and  so  on.  Inputs  needed  are  reliability  prediction,  equipment 
design  options  and  equipment  cost  estimates.  Its  function  is  to 
convert  the  maintenance  concept  to  the  maintenance  plan  in  an 
iterative  process  as  increasingly  refined  and  stable  data  become 
available.  In  the  Air  Force,  the  standard  reference  for  ORLA 
procedures  is  AFLCM/AFSCM  800-4,  "Optimum  Repair  Level  Analysis". 

The  Navy  uses  MIL-STD-1 390,  "Level  of  Repair"  which  will  ultimately 
become  a tri- service  document,  replacing  AFLCM/AFSCM  800-4  for 
Air  Force  use. 

MAINTAINABILITY  APPORTIONMENT 

Maintainability  apportionment  or  allocation  is  analogous  to 
reliability  apportionment  and  is  the  analytical  method  by  which  a 
system  maintainability  requirement  is  distributed  or  allocated  to 
subassemblies,  subsystems  and  components.  It  requires  inputs 
from  a reliability  prediction,  and  is  performed  by  the  system 
contractor  to  establish  numerical  maintainability  requirements 
for  his  suppliers  or  subsystem  designers. 

FAILURE  MODES  ANALYSIS 

Failure  modes,  effects  and  criticality  analysis  (FMECA) 
determines  the  affects  of  a failure  on  a system  or  equipment,  including 
chain  reaction  failures.  Its  use  in  reliability  engineering  was 
discussed  earlier.  In  maintainability  engineering,  it  helps 
establish  failure  detection  logic,  test  points,  and  test  procedures, 
which  in  turn  affect  equipment  design,  test  equipment  requirements, 
maintenance  procedures  and  technical  manuals. 

Methodology  is  described  in  proposed  MIL-STD-1629,  "Procedures 
for  Performing  a Failure  Mode,  Effects  and  Criticality  Analysis," 
which  is  in  the  coordination  cycle  as  this  is  written. 

All  of  these  analysis  tasks  are  relatively  inexpensive  insurance 
against  the  discovery  of  disruptive  maintenance  problems  during 
system  tests.  Later  in  development,  major  design  changes  are 
expensive  and  upsetting  to  the  program,  and  management  is  tempted 
with  undesirable  design  compromises.  Lack  of  good  analysis  can 
lead  to  a patchwork  approach  to  final  system  design.  Good  analysis 
Is  both  an  Air  Force  requirement  and  a prudent  investment  of 
development  money. 
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It  should  also  be  noted  that  the  results  of  the  preceding  analyses 
are  required  inputs  to  Logistics  Support  Analysis,  such  as  described 
in  MIL- STD-1 388,  and  to  Life  Cycle  Cost  Analyses.  (Note:  Paragraph 
5.3  and  subparagraphs  of  MIL-STD-1388  provide  a detailed  listing  of 
items  that  should  be  addressed  by  maintainability  analyses). 

DESIGN  FOR  MAINTAINABILITY 

The  task  of  designing  for  maintainability  consists  of  defining 
specific  hardware  layout  and  packaging  configuration  which  will 
implement  the  design  guidelines  derived  through  maintainability 
analyses.  This  task  gets  into  the  details  of  hardware  design  such  as 
module  configuration  and  arrangement,  choice  of  built-in  failure 
indicators,  electric  cable  layout,  connector  selection,  fastener 
selection  and  placement,  tubing  layout,  circuit  board  layout,  access 
panel  placement,  test  point  selection  and  access  (MIL- STD-41 5) , 
grease  fitting  placement,  materials  selection  for  easy  maintenance, 
design  for  safety  and  other  human  factors,  and  many  more. 

Of  course,  the  design  must  comply  with  standard  military  design 
specifications  called  out  in  the  contract.  Many  provisions  in 
these  standard  design  specifications  have  been  inspired  by  the  need 
for  maintainable  systems  and  equipments.  In  general,  the  designer 
should  follow  good  engineering  design  practices  for  easy  and  economical 
maintenance.  Guidelines  to  good  design  practice  can  be  found  in  a very 
general  outline  form  in  MIL-STD-470.  More  specific  guidelines  for 
electronic  equipment  design  can  be  found  in  the  AFSC  Design  Handbooks 
DH  1-8,  Microelectronics  and  DH  1-9,  Maintainability,  available  through 
the  Aeronautical  Systems  Division  of  AFSC  and  RADC-TR-74-308,  "Main- 
tainability Engineering  Design  Notebook,  Rev  II  and  Cost  of  Maintaina- 
bility" (in  3 volumes).  Also  specific  maintainability  design  criteria 
are  contained  in  MIL-STD-1472,  "Human  Engineering  Design  Criteria  for 
Military  Systems  Equipment  and  Facilities".  Design  guidelines  and 
requirements  for  all  classes  of  equipment  can  be  found  in  the  standard 
military  design  specification  for  those  specific  classes.  Of  interest 
to  testability  is  RADC-TR-78-224,  "A  Design  Guide  for  Built-in-Test." 

As  mentioned  before,  the  results  of  this  detailed  design  work  may  lead 
to  a re-evaluation  of  the  maintainability  requirements,  the  maintenance 
concept,  and  the  maintainability  analyses.  Design  is  part  of  the  main- 
tainability iteration  cycle  and  must  settle  down  to  a fixed  configurati 
by  the  end  of  full  scale  development. 

The  results  of  design  will  be  documented  in  the  contractor  prepared 
design  specification  and  drawings,  and  the  technical  manuals 
mentioned  below. 

As  with  the  design  for  reliability,  it  should  not  cost  much  more  to 
design  a highly  maintainable  system  than  a poorly  maintainable 
system  (though  the  mechanization  could  indeed  be  more  costly).  The 

Note:  RADC-TR-78-224  (A069384) 

RADC-TR-74-308,  Vols  I - III  (A009043) (A005^44) (A009045) 
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early  and  proper  attention  to  these  details  will  result  in  an  overall 
savings  in  development  costs  by  eliminating  disruptive  redesign  work 
which  would  otherwise  be  necessary  when  maintainability  problems  are 
discovered  during  system  tests. 

MAINTAINABILITY  PREDICTION 

Maintainability  prediction  is  an  analytical  effort  performed 
by  the  system  designer.  These  predictions  estimate  the  MTTR  of  the 
system  or  equipment  and  show  the  potential  of  a certain  design  for 
meeting  maintainability  requirements.  Initially,  these  predictions 
will  be  based  on  rough  estimates  from  contractor  experience  with 
certain  equipment  layouts,  but  as  design  progresses  the  predictions 
will  get  more  reliable  as  specific  maintenance  details  evolve. 

Maintainability  predictions  will  be  presented  in  the  reliability 
and  maintainability  allocations,  assessments  and  analysis  report 
(Standard  Data  Item  DI-R-3535  in  the  DoD  Acquisition  Management 
Systems  and  Data  Requirements  List).  When  periodic  R&M  control 
status  reports  are  required  the  current  maintainability  prediction 
should  be  included  for  trend  visibility.  Prediction  methods  are 
presented  in  flIL-HDBK-472,  "Maintainability  Prediction." 

MIL-HDBK-472 , though  still  the  DoD  standard,  was  published  in  1966, 
arid  there  is  some  concern  that  the  methods  are  not  completely 
appropriate  to  modern  technology.  A recently  devel  'ped  maintain- 
ability prediction  method  is  presented  in  RADC-TR-78-169  (A059753), 
"Maintainability  Prediction  and  Analysis  Study."  This  will  ulti- 
mately be  incorporated  into  MIL-HDBK-472.  It  should  also  be  noted 
that  MIL-HDBK-472  presents  four  methods  for  maintainability  pre- 
diction and  the  particular  method  desired  must  be  specified. 

MAINTAINABILITY 
(AJAX  SYSTEM) 


hMHR  PREDICTION 


DESIGN  REVIEW 


The  formal  Air  Force  design  review  will  cover  all  the  ele- 
ments of  the  maintainability  development  cycle  (requirements, 
concept,  analyses,  design,  tests)  at  a suimary  level  of  detail. 

As  in  the  case  of  reliability,  this  is  the  best  opportunity  for 
an  overall  engineering  critique  and  management  appraisal  of  the 
program  progress.  The  Air  Force  program  manager,  his  deputy  for 
logistics,  and  his  technical  advisors  must  personally  participate 
in  this  review.  Representatives  from  the  using  and  supporting 
commands  should  also  participate  in  this  review. 

As  in  the  case  of  reliability  engineering,  informal  reviews 
should  be  arranged  between  the  contractor  and  Air  Force  engineers 
in  advance  of  the  formal  review.  This  will  help  insure  efficient 
conduct  of  the  formal  review. 

DESIGN  SPECIFICATIONS 

Design  specifications  tell  the  designer  what  ground  rules 
or  criteria  he  must  follow  in  coming  up  with  a specific  hardware 
design  to  meet  performance  requirements.  Initially,  the  design 
specifications  will  be  rather  general  Air  Force  prepared 
specifications,  and  will  reflect  only  the  overall  concept 
envisioned  together  with  the  relevant  military  equipment  design 
standards.  As  the  contractor's  maintainability  analysis  and 
design  work  progresses,  he  will  develop  more  detailed  Part  II 
product  specifications  which  reflect  the  results  of  those  studies. 
These  specifications  are  not  a separate  m? inta inability  program 
data  item  delivered  under  the  contract.  Instead,  maintainability 
design  parameters  are  incorporated  into  the  design  specification 
for  each  hardware  element  along  with  design  parameters  inspired 
by  other  engineering  considerations.  Of  course,  maintainability 
design  parameters  must  also  be  included  in  specifications  supplied 
to  subcontractors  and  vendors.  Testability  considerations, 
such  as  built-in-test  requirements  must  also  be  included  in  the 
product  specifications. 

ACCEPTANCE  SPECIFICATIONS 

Acceptance  specifications  prepared  for  use  at  the  end  of 
validation  and  full-scale  development,  must  include  maintain- 
ability demonstration  tests  to  verify  achievement  of  the  specified 
requirements.  We  should  note  that  maintainability  requirements 
can  be  specified  in  various  ways  such  as:  the  mean-time- to-perform 
corrective  maintenance;  the  maximum  time  in  which  a specified 
percentage  of  all  failures  must  be  repaired;  the  median  time  for 
all  repair  activities;  the  percentage  of  repairs  which  can  be 
performed  in  a specified  time;  average  system  downtime  including 
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corrective  and  preventive  maintenance  in  a specified  period  of 
time  or  as  a percentage  of  total  operating  time;  maintenance 
man-hours  per  flight  hour;  and  any  combination  of  the  above.  The 
exact  measures  are  not  critical  if  they  are  understood  and 
satisfactory  to  the  Air  Force  procuring,  using,  and  support 
agencies  and  adequately  defined  in  the  specification.  They  must 
also  be  translatable  to  the  standard  maintainability  reporting 
terms  required  by  AFR  80-5. 

The  acceptance  test,  however,  must  correspond  to  the  form  of 
the  requirement.  MIL-STD-471,  "Maintainability/Verification/ 
Demonstration/Evaluation",  provides  a variety  of  test  plans  to 
measure  different  maintainability  parameters.  Other  tests  not  in 
MIL-STD-471,  such  as  tests  on  the  effectiveness  of  built-in  test 
equipment  or  support  equipment,  must  be  formulated  and  incorporated 
into  the  acceptance  specification.  The  next  revision  of  MIL-STD-471 
will  contain  methods  for  evaluating  fault  detection  and  isolation 
capabilities.  This  should  be  available  late  in  1979. 

DETAILED  MAINTENANCE  PLAN 

The  Detailed 
Maintenance  Plan  is  the 
principal  long  range  I" 
maintenance  planning 
document  which  contains 
the  most  up-to-date 
conclusions  derived 
in  part  from  all  steps 
in  the  maintainability 
development  cycle.  Tha^ 
is,  it  includes  require- 
ments, general  support 
concept,  modular 
configuration  of  the 
system  or  equipment, 
maintenance  approach, 
support  equipment, 
facility  and  personnel  needs,  and  many  other  maintenance  considerations 
which  have  been  developed  from  the  design  requirements.  This  plan 
will  be  incorporated  into  the  Integrated  Logistics  Support  (ILS) 

Plan  developed  by  AFLC  for  the  entire  system. 

Since  this  maintenance  plan  is  of  prime  interest  to  AFLC,  the  Deputy 
Program  Manager  for  Logistics  (DPML)  in  the  Program  Office  should 
directly  participate  in  defining  this  data  item.  As  contract  work 
progresses  from  phase  to  phase,  this  document  will  grow  in  detail  and 
redibility,  and  finally  be  integrated  into  the  total  ILS  Plan.  If 
total  ILS  Plan  is  a contract  data  item  at  the  beginning  of  the 
-''^raln.  this  Integration  could,  of  course,  be  done  then. 


82 


T 


i 


i 

f 

f 


AFR  800-8,  "Integrated  Logistics  Support  Program  for  Systems  and 
Equipment",  explains  policies  and  responsibilities  for  ILS.  AFR 
66-14,  "Equipment  Maintenance  Polices,  Objectives  and  Responsibi 1 i tes" , 
and  AFSC  Supplement  1,  address  the  more  specific  maintenance  policies 
and  responsibilities  which  form  a part  of  ILS.  These  documents 
should  be  consulted  before  development  of  the  Detailed  Maintenance 
Plan.  An  extensive  description  of  the  complete  range  of  ILS 
activities  is  found  in  AFP  800-7,  "Integrated  Logistic  Support 
Implementation  Guide  for  DoD  Systems  and  Equipments".  It  is  an 
all  service  document,  and  is  available  from  the  U.  S.  Government 
Printing  Office. 

DATA  SYSTEM 


A data  system  is  as  important  to  maintainability  engineering 
as  it  is  to  reliability  engineering.  The  general  data  system 
requirements  discussed  for  reliability  also  apply  here.  That  is, 
the  system  needs  to  be  closed  loop  (test  results  recorded,  problems 
analyzed,  and  actions  implemented),  and  should  be  compatible  with 
the  AFM  66-1  maintenance  data  system  so  that  development  program 
data  can  be  compared  later  with  operational  fi^d  data.  In  fact, 
the  data  requirements  for  both  reliability  and  maintainability 
should  be  integrated  into  a single  data  system  for  efficiency. 

Analysis  of  maintainability  data  during  development  is  more 
difficult  than  analysis  of  reliability  data,  since  results  are  so 
strongly  dependent  upon  support  equipment  and  personnel.  During 
the  validation  phase,  the  field  test  equipment,  technical  manuals, 
and  technicians  may  not  be  available  and  maintenance  results  could 
be  far  better  or  far  worse  than  in  deployment.  Hence,  a true 
picture  of  maintainability  progress  is  trickier  to  obtain  than 
a measure  of  achieved  reliability.  On  the  other  hand,  maintenance 
problems  during  validation  and  full  scale  development  provide 
valuable  data  for  improving  test  equipment  and  technical  manuals, 
as  well  as  the  basic  hardware.  Therefore,  the  data  system  is 
essential  even  though  maintainability  estimates  derived  from  its 
data  may  be  rough. 

TECHNICAL  MANUALS 


A major  product  of  the  maintainability  program  is  data  for 
preparation  of  technical  manuals.  These  manuals  will  be  used 
by  operational  support  people  for  system  maintenance  including 
calibration,  repair,  and  preventive  maintenance.  They  must  be 
written  to  match  the  expected  skill  levels  of  personnel  in  the 
field,  intermediate,  or  depot  organizations.  These  manuals  will 
be  affected  by  every  part  of  the  maintainability  program. 
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MAINTAINABILITY  ACCEPTANCE  TESTS 
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Maintainability 
demonstration  or 
acceptance  tests 
are  very  important 
at  the  end  of  the 
validation  phase 
and  are  critical 
at  the  end  of  full 
scale  development. 

They  provide  the 
incentive  for  a 
contractor  to  pur- 
sue an  effective 
maintainability 
program,  and  also 
provide  the  last 
chance  for  the  Air 
Force  to  uncover 
and  correct  any 
deficiencies  be- 
fore the  system 
reaches  the  field. 

Test  procedures 
are  found  in  MIL- 
STD-471 , Mai ntai nabi 1 i ty  Veri f i cati on/Demonstrati on/Eval uation . 

Unlike  reliability  demonstration,  maintainability  demonstration 
does  not  require  long  time  periods.  Failures  are  simulated  by 
introducing  faults  into  the  system,  and  the  times  needed  to  restore 
the  system  are  recorded.  Hence,  even  for  a complex  system,  a 
maintainability  demonstration  can  be  run  in  a few  weeks 
rather  than  several  months  often  required  for  reliability 
verification.  While  we  have  time  economy,  there  is  always 
the  troublesome  question  of  whether  faults  induced  truly 
represent  those  which  will  be  encountered  in  the  field. 

These  faults  or  failure  modes  are  selected  for  simulation 
using  reliability  predictions  to  identify  the  most  likely 
cases.  Even  assuming  this  is  a good  selection,  there  are 
limitations  to  the  simulation  process.  Intermittent  failures, 
for  example,  cannot  be  easily  simulated  and  these  are  extremely 
troublesome  to  repair.  Hence,  the  recording  of  maintenance 
times  for  actual  failures  encountered  in  other  system  tests  is 
a good  procedure,  and  may  be  used  to  supplement  these  maintain- 
ability test  results.  If  a lengthy  and  controlled  system  test 
is  planned  for  some  other  purpose  in  the  program,  it  may  be 
possible  to  utilize  that  test  to  verify  the  achievement  of  the 
maintainability  requirements  at  the  same  time. 


I have  just  simulated  a typical 
failure  out  in  the  field. 

Your  job  is  to 
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The  capability  of  the  fault  detection  and  isolation  (FD/I) 
features  may  also  be  determined  from  data  generated  during 
maintainability  and  other  system  tests.  Methodology  will  be 
available  in  the  next  revision  of  MIL-STD-471 , expected  in 
late  1979.  Until  this  is  available,  each  program  office  must 
create  its  own  procedure  for  determining  FD/I  capability  and 
incorporate  it  into  its  test  plans. 

RESOURCE  REQUIREMENTS 

As  we  have  seen  above,  the  general  drift  of  the  maintain- 
ability program  is  like  the  reliability  program,  except  that 
human  factors  play  a much  bigger  role  (procedures,  hardware 
layout,  skill  levels,  technical  manuals,  equipment  handling, 
etc. ) 

As  in  the  reliability  case,  the  maintainability  program  must 
be  tailored  to  the  specific  hardware  development  program  by 
experienced  maintainability  experts.  The  program  manager 
should  not  attempt  to  simply  reference  MIL-STD-470  in  the 
Statement  of  Work,  since  many  options  are  open  and  the  Air 
Force  needs  to  lay  out  the  ground  rules  from  the  start.  He 
will  need  expert  help  for  preparing  requirements,  selecting 
military  specifications  and  tests,  reviewing  progress,  monitoring 
tests,  etc. ) 

TECHNICAL  MANPOWER 

The  previous  discussion  on  reliability  technical  manpower 
also  applies  here  simply  by  replacing  the  word  reliability  used 
there  with  maintainability.  Briefly,  the  program  manager  should 
have  his  own  in-house  maintainability  manager,  but  that  person 
in  turn  needs  help,  especially  on  large  programs  or  to  fill  gaps 
in  experience.  Help  can  be  obtained  from  Air  Force  laboratories, 
non-profit  engineering  support  contractors,  or  other  contractors 
with  no  competitive  interests  in  the  hardware  to  be  developed. 

The  use  of  experienced  maintenance  personnel  from  AFLC  and  the 
using  command  as  technical  advisors  should  not  be  overlooked. 

REFERENCE  PUBLICATIONS 

There  are  some  good  commercial  reference  books  on  maintain- 
ability engineering,  but  the  R&M  manager's  library  should  include 
at  least  the  following  Government  publications  which  have  been 
referenced  in  the  preceding  discussions  (in  addition  to  AFR  80-5): 

*AFR  66-14,  "Equipment  Maintenance  Policies,  Objectives 
and  Responsibilities,"  and  AFSC  Supplement  1.  This  delineates 
Air  Force  maintenance  program  policies  and  responsibilities,  and 
outlines  specific  considerations  to  be  included  in  developing 
maintenance  concepts  and  plans. 
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*AFR  800-8,  "Integrated  Logistics  Support  (ILS) 

Program  for  Systems  and  Equipments."  This  gives  policy  and 
responsibilities  for  ILS  throughout  the  life  cycle  of  systems 
and  equipments. 

*MIL- STD-470,  "Maintainability  Program  Requirements 
(for  Systems  and  Equipments)."  It  describes  and  discusses  the 
maintainability  program  elements  corresponding  essentially  to 
our  outline  above. 

*MIL-HDBK-472,  "Maintainability  Prediction."  It  gives 
maintainability  prediction  methods. 

*RADC-TR-78-169,  "Maintainability  Prediction  and 
Analysis  Study."  It  provides  a new  prediction  method  not  yet 
Incorporated  into  MIL-HDBK-472. (A059753) 

*MIL- STD-471 , "Maintainability  Verification/Demonstra- 
tion/Evaluation." It  defines  the  various  demonstration  test  plans. 

*AFSC  OH  1-9,  "Maintainability  Design  Handbook."  This 
is  one  of  a series  of  AFSC  design  handbooks  developed  under  the 
supervision  of  Aeronautical  Systems  Division.  It  discusses  main- 
tainability factors,  design  considerations  and  demonstration 
tests . 


*AFLCM/AFSCM  800-4,  "Optimum  Repair  Level  Analysis 
(ORLA)."  This  describes  ORLA;  shows  the  methodology  and  gives 
examples. 


*AFP  800-7,  "Integrated  Logistic  Support  Implementation 
Guide  for  DoD  Systems  and  Equipments."  This  describes  the 
evolution  of  an  ILS  program,  what  it  is,  its  relationship  to 
other  program  elements,  and  the  program  manager's  responsibilities. 
This  is  an  all  service  document  available  from  the  U.S.  Government 
Printing  Office. 

*MIL- STD-1338,  "Logistics  Support  Analysis."  This 
standard  establishes  the  requirements  for  Logistics  Support 
Analysis  applicable  to  both  the  Government  and  contractors. 

♦AFSCP  800-21,  "A  Guide  for  Program  Managers: 
Implementing  Integrated  Logistics  Support."  The  title  is  self- 
explanatory. 

*RADC-TR-78-224,  "A  Design  Guide  for  Built-in-Test." 

The  title  is  self-explanatory.  (Available  in  NTIS). 

In  addition  to  the  above  list  of  publications,  various  textbooks 
on  maintainability  are  available,  and  technical  papers  on 
particular  aspects  are  contained  in  the  proceedings  of  the  Annual 
Reliability  and  Maintainability  Symposium. 
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While  many  computer  programs  have  surely  been  written  by 
contractors  to  carry  out  routine  maintainability  computations, 
there  are  no  standard  routines  in  wide  use. 

FACILITIES 

Facilities  required  for  maintainability  engineering  are  not 
extensive.  Access  to  a computer  is  helpful  in  performing  some 
of  the  analytical  studies,  but  other  than  this,  no  special 
facilities  are  required.  Maintainability  testing  may  sometimes 
require  a mock-up  of  the  aircraft  or  missile  in  which  a subsystem 
is  installed,  or  test  equipment  which  simulates  other  systems 
that  interface  with  the  subsystem  on  test.  Otherwise,  no  special 
facility  is  required.  Maintenance  tools  and  test  equipment 
proposed  for  use  with  the  system  in  the  field,  spare  parts,  and 
technical  manuals  will,  of  course,  be  required. 


SYNOPSIS 


Hardware  maintainability  engineering  consists  of  an  orderly 
sequence  of  steps  strongly  influenced  by  human  engineering  and 
logistic  support  considerations.  The  elements  of  a standard 
program  are  outlined  in  MIL-STD-470,  but  are  subject  to  inter- 
pretation, and  might  be  executed  with  various  levels  of  skill 
and  thoroughness  by  contractor  personnel.  Therefore,  the  Air 
Force  program  manager  should  have  expert  advice  to  prepare 
statements  of  work  and  to  monitor  execution  of  the  tasks.  All 
elements  of  the  program  must  be  fully  coordinated  with  both  the 
using  and  support  cotimands  who  will  be  doing  the  maintenance. 

The  most  critical  elements  are  identification  of  realistic 
requirements,  thorough  analysis  and  design,  evolution  of  a 
good  maintenance  plan,  and  demonstration  of  achieved  results 
through  realistic  acceptance  tests. 
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Chapter  6 

RELIABILITY  AND  MAINTAINABILITY 
MANAGEMENT  BY  ORGANIZATION  LEVELS 


INTRODUCTION 

The  two  pre- 
ceding chapters 
explained  the 
elements  of  a 
reliability  as- 
surance program 
and  a maintain- 
ability assur- 
ance program  in 
the  context  of 
system  develop- 
ment program 
planning  and  ex- 
ecution. Those 
chapters  sand- 
wiched together 
a variety  of  en- 
gineering and 
management  (or 
combination) 
tasks  in  a log- 
ical sequence  as 
they  might  appear 
in  a system  or 
equipment  program 
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plan,  and  cut  across  many  organization  levels. 


In  the  present  chapter,  the  viewpoint  is  different.  Here,  the  over- 
all Air  Force  program  for  R&M  assurance  in  system  or  equipment  de- 
velopment is  viewed  at  separate  organization  levels  and  focuses  on 
the  management  and  contractual  highlights  at  those  levels.  While 
managers  must  understand  the  logic  of  the  R&M  program  structures 
explained  before,  the  present  chapter  homes-in  on  the  principal 
management  guidelines  at  a given  organization  level  which  will  help 
insure  success.  Whether  you  work  in  headquarters  staff,  the  program 


office  or  in  the  contractor's  plant,  there  will  be  something 
here  which  helps  achieve  Air  Force  R&M  goals  through  your  or- 
ganization. The  guidelines  will  not  be  subtle  or  revolutionary, 
but  basic  necessities  which  are,  nevertheless,  easily  overlooked 
or  underplayed. 

The  management  discussions  treat  reliability  and  maintainability 
collectively,  with  some  exceptions,  and  the  discussions  are  or- 
ganized according  to  the  traditional  management  functions  of 
planning,  organizing,  manning,  leading  and  controlling  as  ex- 
plained in  Chapter  3.  At  the  end,  the  subjects  of  warranties 
and  other  contract  incentives  are  briefly  discussed  separately. 

R&M  MANAGEMENT  IN  HEADQUARTERS  STAFF 

This  discus- 
sion applies  gen- 
erally to  the  R&M 
staff  specialists 
at  Hq  USAF  and 
AFSC  who  are  pro- 
viding staff  sup- 
port to  manage- 
ment directors  of 
the  various  head- 
quarters organi- 
zations respons- 
ible for  equip- 
ment and  systems 
development. 


PLANNING  FOR  R&M 

Planning  consists  of  defining  objectives  and  then  developing 
policy,  strategy,  organization,  procedures,  etc.  for  achieving 
them. 

Objectives,  policy,  organization  and  procedures  for  implementation 
of  the  R&M  program  are  well  documented  and  explained  in  the  follow- 
ing publications: 

* AFR  800-2  and  AFSC  Supplement  1, 

Acquisition  Program  Management 

* AFR  800-3,  Engineering  for  Defense  Systems 
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* AFR  80-5  and  AFSC  Supplement  1,  Air  Force  Reliability 
and  Maintainability  Program 

* MIL-STD-785,  Reliability  Program  for  Systems  and 
Equipment  Development  and  Production 

* MIL-STD-470,  Maintainability  Program  Requirements 
(For  Systems  and  Equipments) 

Those  publications  in  turn  reference  many  other  relevant  documents 
dealing  with  Integrated  Logistics  Support  (ILS),  development  test- 
ing, maintenance,  human  engineering,  and  so  on;  however,  the  above 
five  are  the  basic  set  for  R&M  and  systems  program  management  in 
general.  In  the  area  of  general  program  review  and  control,  the 
following  regulations  are  relevant,  but  of  course  apply  to  any 
functional  category  of  program  activity: 

* AFSCR  800-1,  Command  Review  of  Systems  Acquisition 
Programs 

* AFSCR  800-18,  Joint  Operational  and  Technical  Review 
(JOTR) 

The  staff  challenge  here  is  to  keep  all  of  these  documents  current, 
compatible,  and  readable,  with  hopefully  no  conflicts  between  them. 

This  is  especially  challenging  in  the  area  of  maintenance,  logistics, 
and  maintainability  where  there  is  such  a large  collection  of  Air 
Force  and  tri-service  documents  in  being. 

ORGANIZING  FOR  R&M 

The  R&M  staff  responsibility  at  Hq  USAF  for  policy  is  in 
the  Deputy  Chief  of  Staff,  Logistics  and  Engineering,  Directorate  of 
Maintenance  and  Supply,  Engineering  and  Support  Division  (LEYE).  The 
Hq  USAF,  Deputy  Chief  of  Staff,  Resource  Development  and  Acquisition, 
Directorate  of  Development  and  Programming,  Deputy  Directorate  for 
Program  Integration,  Management  Policy  Division  (RDPXM)  is  required 
to  establish  an  R&M  focal  point  for  the  application  of  R&M  in  individual 
programs.  At  Hq  Air  Force  Systems  Command,  the  staff  R&M  focal  point 
is  the  Deputy  Chief  of  Staff,  Systems,  Directorate  of  Acquisition  and 
Engineering  Policy,  Engineering  Management  Division  (SDDE). 

MANNING  FOR  R&M 

In  manning,  the  staff  assures  that  training  programs  provide 
the  qualified  people  needed.  "Qualified"  means  not  only  educated, 
but  experienced.  8oth  AFIT  and  civilian  institutions  are  utilized 
to  build  up  this  career  area. 


LEADING  FOR  R&M 

Tho  is  where  the  staff  level  people  are  extremely  effective. 

The  staff  motivates  and  directs  the  program  managers  to  give  R4M 
the  support  that  is  essential.  This  direction  is  most  effectively 
given  through  the  Program  Management  Directive  (PMD).  That  directive 
xp  icitly  states  the  scope  of  the  R&M  program  expected  and  its 

thlVlm  program  objectives.  It  also  explains 

that  RSM  will  be  a subject  of  detailed  review  at  the  various 
program  review  milestones,  and  that  technical  specialists  will  be 
on  hand  to  review  these  efforts  at  Headquarters.  It  is  in  this 
document  that  the  stage  for  the  RSiM  program  is  set. 

CONTROLLING  FOR  R&M 

Here  is  the  second  area  where  the  staff  level  people  are 
extremely  effective.  Control  is  exercised  through  the  review  and 

approval  of  documents  and 

briefings.  For  example,  the  Program  Management  Plan  (PMP)  which 
responds  to  the  PMD  must  incorporate  R&M  program  plans  which  will 
be  effective,  even  if  the  entire  PMP  is  not  to  be  approved  at 
Headquarters  (only  done  on  a few  major  programs). 

AFR  80-5  details  R&M  program  activities  to  be  performed  and 
requires  review  of  Hq  USAF/LE  of  all  waivers  to  this  policy.  The 

ca"??  no  contract  schedules 

calling  for  full  scale  production  before  notification  R&M  testing 
and  analysis  is  completed.  caLuiy 

various  program  reviews  required  by  AFSCR  800-1  and  800-18 
to  control  and  motivate  R&M  achievements 
Not  oSiv  standardized  R&M  terminology  for  reporting  R&M. 

Not  only  IS  R&M  a review  agenda  item,  but  R&M  specialists  assist 

assessing  the  quality  and  thoroughness  of  the 
reported  R&M  programs. 

SUMMARY 

Qtu  powerful  forces  which  the  staff  exerts  on 

It  leadership  and  control.  This  leadership 

P*”  program.  The  prime 

u-  3nd  control  effort  is  on  the  System  Program 

Manager.  His  personal  attitude  towards  R&M  will  have  a first  order 

shape  of  the  entire  R&M  program,  and  profoundly  affect 
the  life  cycle  cost  of  the  system.  p uuiuiy  arrect 
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R&M  MANAGEMENT  IN  THE  AIR  FORCE  PROGRAM  OFFICE 


The  Air  Force  Program  Office  is  the  critical  focal  point 
in  the  R&M  management  chain.  The  management  emphasis  placed 
on  R&M  by  the  Program  Office  is  far  more  important  to  R&M 
achievement  than  all  the  Air  Force  R&M  standards,  manuals  and 
guidebooks,  because  the  program  manager  in  that  office  has  the 
authority  to  serve  as  the  control  valve  which  governs  the  extent 
of  application  of  those  instructions  (see  AFR  800-2,  Acquisition 
Program  Management).  Even  if  the  Program  Office  places  a good 
set  of  R&M  requirements  in  the  contract,  those  requirements  will 
not  guarantee  success  if  the  contractor  feels  that  the  Program 
Office  does  not  consider  R&M  too  important  relative  to  other 
contract  requirements  such  as  delivery  schedule  or  vehicle  pay- 
load.  In  such  cases,  the  contractor  will  place  his  management 
emphasis  on  items  he  feels  the  Program  Office  will  be  most 
concerned  with,  and  a less  than  adequate  R&M  program  can  easily 
result. 

In  addition,  you  should  expect  a contractor  to  do  all  in  his 
power  to  reduce  his  risks  in  the  R&M  tests,  and  at  the  same 
time,  to  do  all  in  his  power  to  reduce  his  costs.  In  a fixed 
price  contract,  for  example,  reduced  cost  means  increased  com- 
pany profit  and  an  enhanced  commercial  reputation  for  the  con- 
tractor's project  manager.  Also,  in  a cost-reimbursement  con- 
tract, both  achieved  R&M  and  total  project  cost  might  be  used 
in  profit  formulas  as  incentives,  and  the  reduced  profit  from 
lower  achieved  R&M  might  be  outweighed  by  increased  profit  from 
lower  contract  cost.  It  is  the  Program  Office's  responsibility 
to  see  that  contract  incentives  (either  natural  or  created)  are 
not  counterproductive  to  R&M  goals,  and  that  in  any  case,  the 
R&M  program  success  is  not  jeopardized  by  allowing  test  criteria 
which  are  too  lenient  or  R&M  program  tailoring  which  is  too 
skimpy  and  optimistic. 

The  following  sections  discuss  R&M  management  in  the  Air  Force 
Program  Office  with  the  discussion  organized  in  terms  of  the 
management  functions  of  planning,  organizing,  manning,  leading, 
and  controlling.  Planning  is  given  the  most  attention,  since 
it  contributes  to  the  remaining  four  functions  and  must  be  done 
first. 

When  referring  to  the  various  elements  of  R&M  program  below,  only 
highlights  are  noted,  because  the  details  are  explained  in  Chap- 
ters 4 and  5 and  the  reader  should  refer  to  those  chapters  for 
more  insight. 
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PLANNING  FOR  R&M 


Planning  consists  of  identifying  desired  goals  and  then 
delineating  the  best  course  of  action  to  achieve  those  goals. 

R&M  planning  is  not  really  a separate  activity,  but  is  an 
effort  which  must  be  sandwiched  into  overall  planning  for 
the  system.  In  the  conceptual  phase,  for  example,  the  choice 
of  system  design  alternatives  must  include  their  potential 
reliability  and  maintainability  and  attendant  support  costs 
in  order  to  select  the  most  cost-effective  system  alternative. 

In  later  development  stages,  R&M  estimates  are  needed  as  inputs 
for  system  support  planning  for  spare  parts,  depot  facilities, 
training,  etc.  Hence,  R&M  is  a key  element  in  overall  pro- 
gram planning,  and  from  this  planning  should  emerge  a set 
of  realistic  R&M  objectives. 

The  next  phase  of  planning  is  to  construct  an  R&M  program  which 
will  assure  that  those  R&M  objectives  are  actually  achieved. 

The  preceding  discussions  in  Chapters  4 and  5 have  explained 
the  structure  of  such  a program.  Specific  R&M  planning  tasks 
at  the  Program  Office  level  within  the  framework  of  that  struc- 
ture are  explained  in  the  following  paragraphs  in  roughly  chron- 
ological order. 

CONCEPTUAL  PHASE 

Perhaps  even  before  a Program  Office  is  formally  established, 
alternate  system  configurations  to  meet  an  operational  need  are 
envisioned  by  systems  planners.  While  AFSC  may  not  be  concerned 
with  basic  force  trade-offs  such  as  airplanes  versus  missiles, 
(usually  done  at  Hq  USAF  or  DoD  levels),  AFSC  will  be  concerned 
with  trade-offs  within  a particular  vehicle  or  system  configura- 
tion category.  Such  trade-offs  will  include  the  cost  and  ef- 
fectiveness impact  of  performance  parameters  like  altitude,  speed, 
and  range,  and  the  design  of  subsystems  such  as  armament,  elec- 
tronic countermeasures  and  fire  control.  In  these  studies,  the 
potential  reliability  and  maintainability  of  the  total  system  must 
be  considered. 

Reliability  impacts  directly  on  probability  of  mission  success, 
and  indirectly  on  such  items  as  the  weight  of  the  system  if,  for 
example,  redundancy  is  required  to  overcome  a poor  reliability 
potential.  Maintainability  will  also  affect  training  require- 
ments and  support  costs  for  personnel  and  spares.  Both  reliability 
and  maintainability  will  impact  life  cycle  costs.  In  general,  sys- 
tem R&M  estimates  are  necessary  to  identify  the  best  possible  sys- 
tem alternative,  and  to  provide  a valid  picture  of  the  cost-effec- 
tiveness of  the  proposed  system  for  comparison  with  other  system 


alternatives.  Of  course,  both  the  system  using  command  and 
AFLC  must  be  asked  to  provide  their  minimum  acceptable  re- 
quirements . 


R&M  estimates  in  the  conceptual  stage  must  necessarily  be 
based  on  historical  data.  Here,  AFLC  records  in  the  product 
divisions,  records  of  individual  Program  Offices,  and  if  ao- 
plicable,  the  reliability  prediction  formulas  discussed  in 
Chapter  4 are  all  helpful.  Also  helpful  is  an  experienced 
R&M  engineer  who  can  interpret  the  different  ways  R&M  are 
measured.  These  first  estimates  will  begin  the  R&M  planning 
activity,  but  must  be  repeatedly  modified  and  refined  as  more 
data  becomes  available  in  later  phases. 

After  preliminary  system 
trade-offs  are  made  and 
preliminary  R&M  objectives 
are  set,  the  next  activi- 
ties of  the  Program  Office 
are  preparation  of  the  Air 
Force  Program  Management 
Plan,  and  preparation  of 
inputs  to  AFSC  and  Hq  USAF 
for  a Development  Concept 
Paper  assuming  one  is  re- 
quired. 


The  Program  Management 
Plan  (PMP)  is  the  master 
plan  for  achievement  of 
the  program  objectives 
and  is  prepared  by  the 
Program  Office  in  response 
to  a Program  Management 
Directive  from  Hq  USAF 
and  a corresponding  Program  Direction  (AFSC  Form  56)  from  Hq 
AFSC.  R&M  planning  in  the  PMP  document  must  provide  for: 

a.  Definition  and  refinement  of  realistic  quantitative 
R&M  requirements,  to  be  finally  demonstrated  in  the  full-scale 
development  tests. 

b.  Parts  selection  using  military  standard  parts  to  the 
maximum  extent/possible.  Should  the  particular  program  require 
extraordinary  parts  quality  levels  like  in  Minuteman,  the  pro- 
gram must  provide  for  procurement  of  these  special  parts. 

c.  Tracking  R&M  progress  throughout  the  program  to  provide 
a continual  measure  of  achieved  versus  required  R&M. 


DEVELOPMENT 
CONCEPT 
PAPER 
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d.  A planned  period  of  R&M  growth  during  validation  and 
full-scale  development,  using  all  available  failure  and  maintenance 
data  for  R&M  problem  analysis  and  correction  during  this  period. 


e.  Program  review  milestones  for  assessment  of  R&M  progress 
(these  may,  of  course,  be  merged  with  other  review  milestones  as 
appropriate) . 

f.  Adequate  manning  to  insure  competent  R&M  planning  and 
surveillance  of  the  contractor's  efforts,  and  the  possible  need 
to  use  outside  agencies  for  R&M  support. 

g.  Interface  with  the  eventual  using  and  support  commands 
on  R&M  requirements  and  plans. 

The  Development  Concept  Paper  (DCP)  represents  in  effect  a contract 
with  the  Secretary  of  Defense  for  conduct  of  a major  program.  R&M 
performance  "thresholds"  may  be  required  in  the  DCP.  These  are  the 
minimum  performance  limits  and  a DoD  program  review  will  be  triggered 
if  R&M  performance  sinks  below  them.  Hence,  it  is  obviously  important 
that  realism  prevail  in  planning  these  thresholds,  and  in  planning  a 
program  capable  of  meeting  them.  It  is  also  very  important  that  the 
R&M  term  presented  in  APR  80-5  be  used.  These  are  designed  to  prevent 
confusion  between  various  R&M  figures  of  merit  (e.g.,  measures  of 
operational  reliability  versus  measures  of  logistics  demand)  which 
can  cause  even  successful  R&M  achievements  to  appear  dubious. 

VALIDATION  PHASE 


The  Secretary  of  Defense 
approval  of  the  DCP  marks  the 
transition  to  the  validation 
phase  for  major  weapon  systems. 
For  lesser  systems  and  equip- 
ments, approval  will  be  at  a 
lower  level  as  specified  in 
the  Program  Management 
Directive.  Hardware  will  be 
developed  and  tested  by  com- 
peting contractors  in  the  val- 
idation phase,  and  R&M  planning 
will  focus  on  the  contractual 
requirements.  The  statement 
of  work  prepared  by  the  Program 
Office  will  be  written  using 
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AFR  80-5  and  AFSC  Supplement  1,  Reliability  and  Maintainability 
Programs  for  Systems,  Subsystems,  Equipment  and  Munitions,  as 
the  basic  guidance  documents.  The  following  items  are  critical 
inclusions  in  the  statement  of  work; 

a.  Quantitative  R&M  requirements  must  be  specified  and  de- 
fined. It  is  recognized  that  these  requirements  might  not  have 
to  be  achieved  by  the  prototype  hardware  developed  and  tested 

in  the  validation  phase.  However,  the  hardware  must  be  designed 
to  be  inherently  capable  of  achieving  the  required  R&M,  and  R&M 
predictions  should  substantiate  this. 

b.  R&M  testing  is  a must.  This  may  be  evaluation  testing 
or  demonstration  testing  or  both,  but  the  extent  of  the  R&M 
test  program,  its  intent  and,  if  applicable,  the  acceptance 
criteria  must  be  clearly  established. 

c.  Parts  selection  must  be  controlled.  However,  because  of 
difficulties  in  obtaining  preferred  quality  parts  in  small 
quantities,  it  may  not  be  practical  to  fully  employ  them  in 
validation  hardware.  Any  substitute  parts  must  be  identical  in 
form,  fit  and  function  to  the  preferred  parts,  to  preclude 
difficulty  with  including  preferred  parts  in  the  later  systems. 

d.  Fundamental  design  features  which  will  affect  maintain- 
ability must  be  evaluated.  For  example,  built-in  test  provisions 
must  be  included  in  the  validation  phase  equipment  in  order  to 
evaluate  its  functional  effectiveness,  even  though  the  exact  phys- 
ical makeup  of  the  hardware  may  not  correspond  to  operational 
standards . 

e.  R&M  design  trade-off  studies  need  to  be  performed.  These 
include  design  for  reliability,  design  for  maintainability,  redun- 
dancy options,  optimum  repair  level  analysis,  failure  modes  analy- 
sis, and  any  others  required  to  optimize  the  design  or  to  provide 
input  for  other  plans  such  as  the  Detailed  Maintenance  Plan  or  ILS 
Plan. 

f.  R&M  predictions  must  be  continually  refined  as  the  design 
progresses,  to  provide  an  indication  of  potential  R&M  for  use  in 
making  a full-scale  development  decision. 

g.  A closed  loop  data  system  is  required  for  obtaining  R&M 
data  from  all  tests  performed.  This  data  will  then  be  used  to 
determine  the  cause  of  R&M  problems  and  formulate  corrective  ac- 
tion. 
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h.  Program  and  design  reviews  are  essential  for  control 
and  motivation  of  the  entire  R&M  program,  and  to  insure  that 
detailed  R&M  design  effort  is  progressing  in  a professional 
way. 


i.  Appropriate  deliverable  data  items  must  be  selected 
to  give  the  Air  Force  Program  Office  needed  visibility  into 
the  above  activities  and  document  results. 

The  above  list  of  validation  phase  statement  of  work  provisions 
represent  planning  highlights  never  to  be  overlooked.  More  de- 
tails and  explanations  have  been  covered  in  Chapters  4 and  5 
and  in  APR  80-5  and  MIL-STDs  785  and  470. 

Once  the  statement  of  work  has  been  prepared  and  the  request 
for  contractor  proposal  has  been  issued  in  industry,  source 
selection  of  the  validation  phase  contractors  is  the  next 
critical  activity  which  could  be  classed  as  a planning  function. 
Proposal  evaluations  must  consider  R&M  aspects  of  each  proposal 
to  Insure  that  the  bidder  understands  what  is  required  of  him, 
and  is  both  willing  and  able  to  provide  it.  The  proposals  are 
the  first  place  in  which  a contractor  may  attempt  to  obtain  re- 
laxation of  R&M  requirements,  often  through  very  subtle  use  of 
words  and  through  the  use  of  R&M  jargon  which,  to  the  non-spe- 
cialist, seems  to  promise  more  than  it  does.  Under  the  working 
pressure  of  source  selection,  even  the  experienced  R&M  engineer 
must  guard  against  a tendency  to  assume  too  much.  Questions  for 
clarification  or  proposals  and  later  negotiations  with  bidders 
in  the  competitive  range,  must  resolve  any  uncertainty  and  make 
sure  the  contractor  is  indeed  proposing  the  R&M  program  that  the 
Air  Force  desires.  Careful  consideration  of  R&M  during  evalua- 
tion and  negotiations  will  preclude  an  erroneous  conclusion  by 
the  contractor  that  R&M  need  not  be  a great  concern  of  his  dur- 
ing the  program,  and  of  course  will  reduce  the  potential  for  dis- 
putes later  on.  This  may  seem  a minor  point,  but  there  have  been 
many  programs  in  which  R&M  was  pursued  with  a "low  profile"  even 
though  contract  provisions  seemed  complete,  and  such  a profile 
began  to  form  during  negotiations  with  the  bidders.  Contractors 
will  utilize  negotiation  issues  to  gauge  Air  Force  Program  Office 
emphasis  on  the  multitude  of  provisions  in  any  Government  contract. 

One  good  way  to  put  early  emphasis  on  R&M  is  to  require  bidders 
to  submit  a Preliminary  R&M  Program  Plan  with  the  proposals  for 
evaluation  by  the  source  evaluation  team.  (This  practice  is 
called  out  in  MIL-STD-470  and  785,  but  its  applicability  to  a 
particular  contract  solicitation  in  the  validation  phase  is  sub- 
ject to  the  judgment  of  the  Program  Manager).  Deficiencies  in 
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the  preliminary  R&M  Program  Plan  will  then  be  the  subject  of  pre- 
contract negotiations,  and  these  deficiencies  must  be  ironed  out 
before  any  contract  is  signed.  If  properly  written,  the  negotiated 
R&M  Program  Plan  can  then  be  incorporated  into  the  contract  and  be- 
come the  basis  for  contractual  compliance.  This  precontractual  ap- 
proach to  the  R&M  Program  Plan  will  insure  that  the  R&M  program 
gets  off  to  a good  start,  with  the  Government  and  the  contractor 
having  a mutual  understanding  of  the  R&M  program  elements  and  the 
ground  rules  for  their  execution. 


Sir,  I know  all  these  specs. 


are  your 
favorites  7 


m. 


In  the  K&M  Program  Plan,  the 
contractor  defines  his  approach 
to  achieving  RiM  requirements, 
his  milestones,  and  his  organ- 
ization. This  plan  Is  very 
Important  since  It  establishes 
the  understanding  between  the 
contractor  and  the  Air  Force 
on  the  R4M  effort  expected 
and  provides  a reference  for 
review  and  control.  Hence, 
this  document  must  reflect 
the  Statement  of  Work  require- 
ments and  completely  describe 
an  adequate  program  to  pursue 
them.  The  approved  R&M  Program 
Plan  (preferably  negotiated 
before  contract  signing)  should 
leave  no  doubts  about  what  will 
be  accomplished.  The  program 
elements  to  be  Included  In  a 
standard  R&M  program  were  described  In 
Guidance  for  the  contractor  In  writing 
the  Data  Item  Description  DIR-R-3533, 
Program  Plan,  listed  In  the  DoD  Acquis 
Data  Requirements  Control  List. 


RELIABILITY 

& 

MAINTAINABILITY 

program 

PLAN 

DI-R-3533 

I Ajax  Company 


Chapters  4 and  5. 
this  plan  Is  given  In 
Reliability /Maintainability 
Itlon  Management  Systems  and 


Another  plan  to  be  prepared  during  the  validation  phase  Is  the 
Preliminary  Integrated  Logistics  Support  (ILS)  Plan.  Air  Force 
Pamphlet  800-7,  Integrated  Logistics  Support  Implementation  Guide, 
discusses  the  ILS  procedure  In  great  detail  and  should  be  consulted 
for  further  information. 


By  the  end  of  the  validation  phase, 
the  Program  Office  must  have 
the  following  R&M  products  In 
hand  In  order  to  make  decisions 
and  plans  for  the  next  develop- 
ment phase: 

a.  Predictions  of  the 
potential  R&M  of  the  system  must 
be  up-to-date.  These  must  be 
realistically  derived  and  com- 
mensurate with  the  expected 
operational  environment  and 
the  selected  parts  quality.  An 
historical  record  of  these  pre- 
dictions should  have  been  con- 
tinuously updated  through  the 
validation  phase. 


TAKE  OFF  CHECKLIST 
FOR  FULL  SCALE 
DEVELOPAAENT  i 

Bf  PREDICTIONS 
(/  ACHIEVED  R/M  DATA 
Bf  SYSTEM  TRADE-OFFS 
0^  DESIGN  SPECS 

□ TEST  SPECS 

□ R&M  PROG  PLAN 
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b.  Achieved  R&M  of  the  validation  hardware  based  on  actual 
test  data  should  be  in  hand.  Most  of  the  time,  the  achieved  R&M 
will  be  significantly  below  the  predicted  and  required  values. 

Hence,  the  Program  Manager  must  have  a track  record  of  R4M  growth 
experienced  during  validation  and  sound  engineering  solutions  to 
all  R&M  problems  found.  As  far  as  possible,  these  solutions 
should  be  tested  and  validated  during  the  validation  phase. 

c.  System  design  trade-off  studies  should  be  complete  using 
realistic  R&M  inputs,  to  define  the  most  cost-effective  system 
configuration. 

d.  System  design  specifications  intended  for  the  full-scale 
development  phase  are  needed.  These  must  incorporate  quantitative 
R&M  requirements  clearly  defined,  and  all  the  corresponding  R&M 
design  requirements  necessary  for  their  achievement,  that  is,  parts 
selection  criteria,  built-in  test  features,  modular  configuration, 
environmental  criteria,  etc. 

y 

e.  System  acceptance  specifications  are  needed  which  define 
R&M  demonstration  tests  to  be  performed  in  the  full-scale  development 
and  production  phases,  including  the  test  plans  and  test  levels, 
system  burn-in  requirements,  ground  rules  for  test  measurements,  ground 
rules  for  classification  of  failures,  and  so  on.  Environmental 
qualification  tests  must  also  be  defined. 

f.  The  R&M  program  plan  for  full-scale  development  and 
production  must  be  completed  by  the  contractor  (or  competing 
contractors)  by  the  end  of  validation.  This  plan  will  then  be  used 
for  negotiating  a follow-on  full-scale  development  contract. 

g.  The  Program  Manager  must  write  an  Air  Force  R&K  Management 
Plan  (AFR  80-5,  Attachment  3,  Paragraph  4e).  This  will  be  submitted 
as  part  of  the  Program  Management  Plan  (PMP).  Its  contents  are 
described  in  AFR  80-5,  Attachment  4. 

DoD  or  Air  Force  approval  to  proceed  to  full-scale  development  will 
be  based  on  assurance  that  system  trade-offs  have  produced  a balanced 
and  realistic  set  of  performance  parameters,  risk  areas  have  been 
identified  and  reduced  to  acceptable  levels,  cost  and  schedule 
estimates  for  full-scale  development  are  reasonable  and  acceptable, 
and  contractual  aspects  are  sound  (AFSCP  800-3).  These  factors 
refer  to  R&M  as  well  as  other  perfonmance  parameters. 
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FULL  SCALE  DEVELOPMENT 


During  full-scale  development  planning,  much  of  the  above 
discussion  of  work  statement  preparation  and  source  selection 
again  applies.  Essential  differences  between  these  phases  are 
that  during  validation,  the  realism  of  R&M  requirements  must  be 
established,  system  design  trade-offs  made,  and  R&M  problems 
identified  and  eliminated,  whereas  during  full-scale  development, 
the  requirements  are  firm,  and  the  program  is  geared  toward  im- 
plementing final  design  decisions  and  proving  through  demonstra- 
tion tests  that  R&M  requirements  will  be  met.  The  Program  Man- 
ager must  again  guard  against  relaxation  of  the  contractor's  ef- 
forts, and  must  plan  to  provide  the  test  time,  equipment  test 
samples,  and  the  facilities  needed  for  R&M  demonstration  tests. 

During  full-scale  development,  the  program  planning  framework 
consists  mainly  of  the  contract  which  in  turn  incorporates  the 
system  design  and  acceptance  specifications,  and  the  negotiated 
R&M  Program  Plan.  During  the  early  part  of  full-scale  develop- 
ment, the  contractor  must  also  prepare  an  R&M  Test  Plan  which 
is  another  key  planning  document. 

The  R&M  Test  Plan  provides  the  execution  details  of  the  R&M 
demonstration  tests.  While  general  ground  rules  are  covered  by 
the  contractual  documents,  the  plan  covers  the  multitude  of  par- 
ticulars which  must  be  defined  before  the  tests  are  run.  It  is 
a potential  source  of  compromise  to  the  intent  of  the  test  re- 
quirements and  careful  review  is  essential.  Data  Item  Descrip- 
tion DI-R-3538,  Reliability/Maintainability  Demonstration  Plan 
gives  particulars  for  writing  this  plan. 

Next  to  unambiguous  requirements  and  the  selection  of  quality 
parts,  the  R&M  tests  are  the  most  essential  element  in  the  R&M 
program  during  the  full-scale  development  phase.  Without  clear 
requirements,  no  program  can  have  a reasonable  chance  of  success. 
Without  quality  parts,  you're  doomed.  Without  a good  test,  there 
is  no  way  of  knowing  what  has  been  accomplished  and  it  is  almost 
certain  that  the  program  will  be  emasculated.  With  clear  R&M  re- 
quirements, quality  parts,  valid  tests,  and  strong  emphasis  by 
the  Program  Office,  a reasonable  and  competent  contractor  is  not 
likely  to  neglect  the  other  program  elements  and  the  R&M  program 
will  almost  surely  be  a success. 
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Finally  during  full-scale  development,  a final  Integrated 
Logistics  Support  Plan  will  be  prepared  utilizing  R&M  inputs 
from  the  Detailed  Maintenance  Plan  (see  Chapter  5). 

PRODUCTION 

In  production,  R&M  activity  will  be  concerned  mainly  with 
finding  and  fixing  problems  arising  during  production.  These 
will  be  primarily  workmanship  and  parts  defects,  since  most  de- 
sign problems  should  have  been  rectified  by  this  time-.  Periodic 
production  reliability  verification  must  be  performed  to  identify 
and  correct  reliability  degradation  during  the  production  run. 
Engineering  change  proposals  (ECPs)  must  be  evaluated  for  their 
effects  on  R&M.  Quality  control  plays  its  most  important  role 
during  this  phase. 

DEPLOYMENT 

Finally,  in  deployment,  field  data  must  be  used  to  track  field 
RiM,  identify  problems  and  determine  fruitful  areas  fgr  R&M  im- 
provements. The  AFLC  Improved  Reliability  of  Operational  Systems 
(IROS)  Program  will  indicate  which  particular  system  components  are 
the  most  costly  to  support.  This  data  may  provide  the  oppor- 
tunity to  design  system  modifications  which  wtll  reduce  support 
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costs.  By  comparing  the  costs  of  the  improvements  with  the 
field  costs  and  potential  savings,  AFLC  and  AFSC  can  determine 
the  economic  attractiveness  of  proposed  system  changes. 

ORGANIZING  FOR  R&M 

Inputs  to  the  R&M  Manager's  activities  are  required, 
not  only  from  AFSC  R&M  engineering  specialists,  but  also 
from  the  using  and  supporting  commands.  Both  the  using 
and  support  commands  are  a source  of  R&M  requirements  and 
experience  data.  Representatives  of  these  comnands  will 
also  review  R&M  progress  and  gather  planning  information. 

The  realism  of  R&M  objectives,  at  least  in  early  program 
stages,  will  be  based  significantly  on  AFLC  field  data 
banks.  The  Program  Office  must  therefore  establish  good 
working  relationships  with  these  agencies. 

The  R&M  Manager  will  also  require  the  services  of  various 
specialists.  One  of  the  most  important  is  the  parts  spe- 
cialist. Even  in  the  best  of  programs  it  is  probable  there 
will  be  requests  for  the  use  of  parts  not  on  Air  Force  pre- 
ferred parts  lists.  Sometimes  a ne^/  type  of  part  is  required, 
some  parts  seem  to  resist  standardization,  and  often  a con- 
tractor may  not  be  aware  of  a preferred  part  that  will  meet 
his  needs.  Difficulties  in  obtaining  preferred  parts  and 
the  obsolescence  of  parts  on  the  preferred  listing  add  to 
the  problem.  It  is  the  role  of  the  parts  specialists  to  re- 
view parts  lists,  and  especially  requests  for  nonstandard 
parts,  to  accomplish  the  following: 

a.  Determine  if  a preferred  part  may  be  used  in  place 
of  a proposed  part. 

b.  Recommend  quality  assurance  procedures  for  nonstandard 
parts. 

c.  Identify  acceptable  replacements  for  standard  parts 
which  are  obsolete. 

d.  Assist  in  locating  sources  for  preferred  parts. 

The  assistance  of  parts  specialists  is  available  from  the  De- 
fense Electronics  Supply  Center,  Dayton,  Ohio,  for  all  types 
of  electrical  and  electronic  parts,  and  from  Rome  Air  Develop- 
ment Center,  Griffiss  AFB  NY,  for  microcircuits,  semiconductors. 
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printed  circuit  boards  and  their  associated  connectors.  Assistance 
on  selection  of  mechanical  parts.  Including  nuts  and  bolts  Is  avail- 
able from  the  Defense  Industrial  Supply  Center,  Philadelphia  PA. 

The  RiM  manager  of  a program  Involving  electronics  must  arrange  for 
an  Interface  with  those  organizations. 

The  RSM  manager  will  also  require  the  use  of  environmental  specialists 
to  assure  proposed  environmental  tests  are  commensurate  with  the 
mission  requirements.  The  support  of  specification  writers  Is  also 
needed  to  see  that  all  referenced  standards  are  up-to-date  and  that 
the  standard  "boiler  plate"  Is  Intact.  Quality  control  specialists 
will  be  needed  to  evaluate  and  negotiate  proposed  quality  programs. 

Of  course,  an  RAM  engineering  specialist  will  be  needed  to  keep  the 
requirements  unambiguous,  evaluate  contractor's  Inputs,  and  translate 
the  technical  jargon  Into  terms  meaningful  to  the  Program  Manager.  This 
specialist  will  also  provide  Information  to  aid  In  test  planning,  such 
as  the  time  required  for  reliability  tests  for  various  sample  sizes 
and  confidence  levels. 

When  the  Program  Manager  does  not  have  available  to  him  a full  time 
RiM  specialist,  his  organization  should  Include  provision  for  part 
time  or  temporary  support  from  other  Air  Force  offices.  These  can 
Include  the  reliability  staff  In  his  own  division  or  center,  the 
services  of  RSM  engineers  from  other  Program  Offices,  or  the  use  of 
specialists  from  other  agencies,  such  as  the  Reliability  Branch  of 
the  Rome  Air  Development  Center. 

Industrial  support  may  also  be  a part  of  the  Program  Manager's  RSM 
organization.  Reliability  specialists  from  industry  have  been  hired 
as  a source  of  manpower  for  such  activities  as  providing  full  time 
test  monitors  for  lengthy  reliability  tests.  They  have  also  been 
used  as  program  monitors  and  proposal  evaluators,  though  these  jobs 
must  be  left  to  Air  Force  personnel  whenever  possible.  In  using 
Industrial  sources,  hardware  manufacturers  should  generally  be 
avoided  with  preference  for  research  firms  not  concerned  with 
manufacturing.  Otherwise  conflicts  of  Interests  and  friction  are 
likely  between  the  system  contractor  and  the  monitoring  contractor. 

MANNING  FOR  R&M 

AFSC  2895  Is  the  Air  Force  Specialty  Code  for  a reliability 
engineer.  An  officer  with  this  AFSC  and  a Master's  degree  in 


reliability  from  the  Air  Force  Institute  of  Technology  (AFIT) 
would  be  a valuable  asset  on  the  Program  Manager's  staff.  Equally 
valuable  would  be  a military  o-'  civilian  Air  Force  member  with 
an  engineering,  physics  or  mathematical  degree  and  several  years' 
experience  in  RiM  support.  While  the  supply  of  such  personnel  is 
increasing,  it  still  falls  short  of  demand.  Hence,  a particular 
Program  Manager  may  be  forced  to  create  his  own  RiM  specialist 
or  rely  on  outside  agency  support  as  discussed  above.  However, 

AFR  80-5  stipulates  that  AFSC  will  provide  an  R&M  support  specialist 
for  each  system  program  or  project. 

There  are  many  sources  of  R&M  education.  Starting  September  1979, 
AFIT  will  offer  a program  leading  to  a Master's  degree  in  Reliability 
Engineering.  AFIT  also  conducts  short  courses  in  R&M  Eiij'ineering, 
one  of  two  weeks  duration  and  a more  detailed  course  of  ’■en  weeks. 
Many  civilian  universities  also  conduct  short  courses  in  R&M 
Engineering  and  the  University  of  Arizona  has  a degree  program  in 
Reliability  Engineering.  Courses  can  be  located  from  university 
circulars  and  from  listings  in  the  Newsletter  of  the  Institute  of 
Electrical  and  Electronic  Engineers  (IEEE)  Reliability  Group.  The 
IEEE  Reliability  Group  also  publishes  quarterly  transactions  of 
technical  papers  on  reliability,  and  with  the  American  Society  for 
Quality  Control  and  several  other  professional  societies  related  to 
product  assurance,  sponsors  a yearly  symposium  of  reliability  and 
maintainability,  held  each  January.  The  proceedings  of  the  symposium 
provide  good  educational  material  for  the  aspiring  R&M  engineer, 
and  are  available  through  most  Air  Force  technical  libraries.  (See 
Appendix  A of  this  guidebook  for  a list  of  schools  and  symposia 
which  provide  R&M  educational  opportunities). 

In  starting  a program,  it  is  advisable  for  the  Program  Manager  to 
seek  experienced  help,  even  on  a temporary  basis,  if  he  does  not 
yet  have  an  R&M  specialist  on  his  staff.  Available  short  courses 
and  selected  literature  reviews  could  .then  be  used  to  bring  a 
designated,  but  inexperienced,  R&M  engineer  up  to  a reasonable 
competence  fairly  rapidly.  On  a long,program,  the  use  of  regular 
university  courses  or  degree  programs  might  be  a good  investment, 
if  there  is  some  reasonable  assurance  the  individual  will  be 
retained. 

For  period  of  high  activity  above  the  capability  of  a normally 
adequate  R&M  staff,  such  as  concurrent  reliability  tests  at 
competing  contractors,  contractual  ^pport  should  be  considered. 
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The  use  of  other  Government  agencies  such  as  in-plant  APPRO  or 
DCASO  representatives  located  at  contractors'  plants  may  also  be 
a useable  source  of  supplemental  manning,  providing  the  in-plant 
representatives  are  able  to  devote  the  necessary  time  and  have 
the  know-how. 

Finally,  it  must  be  recognized  that  the  presence  of  a trained 
R&M  engineer  in  the  office  does  not  relieve  the  equipment  or 
subsystem  engineer  of  his  responsibility  for  the  R&M  of  his 
assigned  hardware.  It  is  the  task  of  the  R&M  engineer  to  manage 
the  R&M  program  and  provide  specialized  support  to  the  equipment 
engineers.  The  equipment  engineers  are  responsible  for  seeing 
the  R&M  aspects  are  accorded  appropriate  consideration  in  the 
design  of  their  equipment  by  the  contractors.  Should  the  equip- 
ment engineer  dismiss  R&M  as  outside  his  responsibility,  likely 
results  are  the  overloading  of  the  R&M  engineer  as  he  tries  to 
independently  impact  all  the  various  equipment  procurements,  a 
degradation  of  the  contractor's  R&M  program  since  he  is  more  like- 
ly to  be  concerned  with  pleasing  the  equipment  engineer  than  the 
R&M  engineer,  and  possibly  a conflict  of  direction  to  the  contrac- 
tor from  the  equipment  engineer  and  the  R&M  engineer.  While  the 
special  skills  of  the  R&M  engineer  are  needed  to  maintain  an 
effective  R&M  program,  the  equipment  engineer  has  responsibility 
for  total  equipment  performance  including  R&M. 

LEADING  FOR  R&M 

The  Program  Manager  must  obviously  lead  the  R&M  effort.  Work- 
ing agreements  must  be  arranged  with  the  other  Air  Force  offices 
involved.  This  will  include  arrangements  for  the  use  of  specialists 
within  the  Program  Manager's  division  staff,  the  establishment  of 
liaison  with  using  and  support  commands,  and  formal  working  agree- 
ments with  the  Air  Force  laboratories.  Defense  Electronics  Supply 
Center,  and  sources  of  field  data  as  appropriate.  The  Air  Force 
team  should  be  in  working  shape,  with  responsibilities  clearly 
understood,  well  in  advance  of  the  first  Source  Selection  Board 
meeting. 

The  statement  of  work  and  contractual  documentation  are  an  essential, 
but  not  sufficient  element  in  directing  the  contractor.  Air  Force 
direction  is  also  supplied  in  official  responses  to  key  contractor 
prepared  data  items  such  as  his  proposal,  R&M  Program  Plan,  predic- 
tion and  test  plans,  a:  well  as  by  Air  Force  responses  to  design  and 
program  reviews. 


Motivation  of  both  Pruyram  Office  personnel  and  the  con- 
tractor for  R&M  progress  will  be  directly  proportional  to 
the  concern  evidenced  by  the  R&M  manager.  Because  the  ef- 
fects of  R&M  program  deficiencies  do  not  become  obvious 
until  the  equipment  is  in  test,  there  is  a tendency  oy 
both  Air  Force  and  contractor  management  to  focus  their 
attention  on  more  immedi'ite  and  obvious  problems.  How- 
ever, when  R&M  problems  do  appear  in  tests,  it  will  be 
expensive  and  it  might  be  impractical  to  correct  them. 

Hence,  the  astute  Program  Manager  will  maintain  an  R&M 
emphasis  throughout  the  program  to  preclude  unpleasant 
surprises  in  its  late  stages. 

CONTROLLING  FOR  R&M 

The  Air  Force  approval  of  contractor  data  items  con- 
stitutes an  important  control  over  the  contractor  R&M  pro- 
gram. When  correction  of  data  deficiencies  and  the  program 
activities  they  represent  is  a prerequisite  for  data  ap- 
proval, the  contractor  must  take  some  action.  However,  the 
Program  Manager  and  his  R&M  advisors  must  be  keenly  aware 
from  the  very  beginning  of  the  program,  that  the  Air  Force's 
claim  of  data  deficiencies  must  be  based  on  explicit  con- 
tractual standards  of  acceptable  data  submission.  The  con- 
tract must  be  explicit  as  possible  in  defining  the  scope, 
procedures,  and  data  sources,  for  studies,  analyses,  and 
plans.  Especially  in  fixed  price  contracts,  or  in  cost  re- 
imbursement contracts  incorporating  a large  target  price 
profit  incentive,  the  contractor  is  strongly  motivated  to 
save  money.  (In  a fixed  price  contract,  every  dollar  saved 
is  a dollar  earned).  Therefore,  the  contractor  is  motivated 
to  provide  analyses  and  reports  with  only  the  minimum  degree 
of  completeness  which  he  thinks  the  Program  Office  will  ac- 
cept. Without  tangible  standards  of  data  completeness  in  the 
contract,  the  Air  Force  R&M  manager's  task  is  frustrated  in 
the  early  analytical  phases  of  the  R&M  program. 

Periodic  design  reviews  provide  another  monitoring  device  for 
checking  the  progress  on  the  R&M  program.  Periodic  reviews 
of  both  the  contractor's  and  the  Air  Force  programs  and  com- 
parison to  previously  established  R&M  milestones  will  also 
help  keep  the  R&M  program  under  control.  During  the  valida- 
tion phase,  frequent  review  of  achieved  R&M,  and  status  of 
corrective  actions  for  known  problems,  will  be  the  key  items 
for  R&M  control . 
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The  most  significant  R&M  control  over  the  contractor  will 
be  the  R&M  demonstrations.  As  previously  mentioned,  the 
loss  of  this  control  is  usually  fatal  to  an  R&M  program. 

Further,  any  compromises  in  R&M  demonstrations  should  be 
considered  only  with  a clear  understanding  and  the  most 
critical  examination  of  the  risks  involved. 

Other  important  control  devices  over  the  contractor’s  R&M 
efforts  are  the  parts  selection  criteria,  parts  quality 
assurance  (screening)  procedures,  and  system  quality  con- 
trol requirements. 

Once  the  equipment  is  in  the  field,  R&M  visibility  is  pro- 
vided by  the  Air  Force  maintenance  data  systems.  While 
visibility  alone  does  not  constitute  control,  programs  like 
IROS  use  this  data  to  focus  manageinent  attention  on  the  items 
with  the  highest  support  costs,  and  thereby  provide  the  man- 
ager with  the  information  he  needs  to  direct  his  R&M  improve- 
ment program. 

Finally,  all  the  standard  management  control  techniques  such 
as  PERT,  PERT-COST,  milestone  charts,  etc.,  should  be  applied 
to  R&M  as  well  as  other  program  elements.  R&M  engineering  is 
not  a separate  program  running  concurrently  with  system  develop- 
ment, but  it  is  an  integrated  part  of  the  overall  program, 

SUMMARY 

The  hallmarks  of  a well -managed  R&M  program  at  the  Air  Force 
Program  Office  can  be  listed  as  follows; 

1.  R&M  'requirements  are  realistic,  and  if  necessary,  are 
updated  to  stay  realistic  until  the  final  system  requirements 
are  fixed  for  full-scale  development. 

2.  The  Program  Office  has  arranged  for  the  support  of  tech- 
nical specialists,  (R&M,  specifications,  parts  control,  environ- 
mental testing). 

3.  Liaison  arrangements  have  been  made  with  the  using  and 
support  commands. 

4.  Appropriate  R&M  program  elements  are  explicitly  defined 
in  contractual  documents,  and  compliance  is  enforced. 

5.  Air  Force  R&M  specialists  review  submitted  R&M  data  items 
and  provide  timely  feedback  to  the  contractor. 


110 


6.  Design  reviews,  contractor  program  reviews,  and 
internal  Program  Office  reviews  include  R&M  program  status, 
the  status  of  R&M  data  items,  and  correction  of  their  de- 
ficiencies . 

7.  Reliability  growth  during  the  validation  phase  is 
planned  and  monitored  by  the  program  manager. 

8.  R&M  demonstration  tests  are  well  defined  and  pro- 
fessionally monitored  by  Air  Force  R&M  specialists. 

9.  Air  Force  preferred  parts  are  used  to  the  maximum 
extent  possible. 

10.  R&M  are  considered  essential  system  parameters  which 
must  be  built  into  the  design,  along  with  other  performance 
parameters. 

R&M  MANAGEMENT  IN  THE  CONTRACTOR'S  ORGANIZATION 


This  discussion  is 
directed  toward  the 
system  or  equipment 
project  manager  in 
the  contractor's  or- 
ganization who  is 
the  counterpart  of 
the  Air  Force  System 
or  Equipment  Program 
Office  Manager.  The 
project  manager  is 
our  focal  point,  be- 
cause he  is  the  per- 
son who  makes  the 
project  management 
decisions  which  ul- 
timately reflect 
the  contractor's 
success  in  achieving 

R&M  objectives.  We  will  make  some  management  suggestions  which 
should  assist  the  project  manager  and  his  company  achieve  the 
reliability  and  maintainability  objectives  which  will  hopefully 
benefit  both  the  company  and  the  Air  Force.  We  will  do  this  in 
the  frame  work  of  the  five  management  functions  explained  in 
Chapter  3;  planning,  organizing,  manning,  leading,  and  control- 
ling. The  discussion  is  not  specialized  to  any  particular  tech- 
nology field  such  as  electronics  or  engines,  but  should  be  ap- 
plicable anywhere. 
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PLANNING  FOR  R&M 

Planning  must  be  started  before  any  other  activity  in  a 
project  can  be  initiated,  since  planning  impacts  all  the 
other  management  functions.  In  general,  planning  consists 
of  identifying  desired  goals  and  then  delineating  the  best 
course  of  action  to  achieve  those  goals.  We  will  separate 
planning  at  the  contractor's  facility  into  two  major  cate- 
gories: those  plans  which  are  independent  of  any  particu- 
lar contract,  and  those  which  are  made  for  the  purpose  of 
executing  a specific  contract.  The  first  category  will 
cover  plans  which  should  be  made  to  insure  that  the  company 
will  be  in  a good  competitive  position  with  respect  to  R&M 
when  a particular  contract  bid  opportunity  comes  along. 

The  second  category  of  plans  will  be  dictated,  for  the  most 
part,  by  R&M  requirements  in  the  contract  which  has  been  won. 

PLANNING  INDEPENDENT  OF  A PARTICULAR  CONTRACT 

First  of  all,  the  project  manager  must  be  sure  he  under- 
stands overall  company  goals.  Perhaps  the  goal  may  be  stated 
as:  "To  successfully  compete  for  Air  Force  development  con- 
tracts, produce  high  quality  products,  and  make  a reasonable 
profit."  While  that  sounds  good,  the  project  manager  will 
need  more  information.  That  is,  what  is  top  management's  at- 
titude towards  the  relative  importance  of  successful  competi- 
tion versus  product  quality  versus  contract  profits?  Is  top 
management  willing  to  reduce  product  quality  in  order  to  main- 
tain a certain  profit  margin?  Is  competitive  position  for 
future  contracts  through  superior  technical  capability  more 
important  than  profits  on  a present  contract?  Are  reasonable 
profits  now  of  paramount  importance  and  future  Air  Force  con- 
tracts secondary?  Will  the  project  manager  receive  a bonus 
based  on  demonstrated  reliability  in  acceptance  tests  or  based 
on  net  company  profits  from  the  project?  In  other  words,  the 
project  manager  needs  to  know  company  goals  and  their  priorities 
so  that  he  can  plan  his  work  accordingly.  He  needs  to  have  a 
feeling  for  the  management  trade-offs  his  superiors  will  view 
with  favor. 

From  the  Air  Force's  view  of  this  example,  hopefully  high  qual- 
ity products  come  first,  capability  for  successful  competition 
comes  second,  and  profits  come  third.  We  can  argue  that  this 
order  of  precedence  is  also  bast  for  the  long  range  interests 
of  the  company,  because  quality  will  lead  to  successful  com- 
petitio..  which  in  turn  will  lead  to  profits.  However,  priorities 
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' often  get  changed  or  are  disregarded  when  the  company  gets  into 

; competitive  bidding  for  Air  Force  contracts.  The  basic  reason 

I for  this  is  that  the  Air  Force  is  required  to  award  contracts  to 

r the  lowest  qualified  bidders.  It  is  clear  that  a contractor  can 

make  his  lowest  bid  if  he  proposes  to  do  that  which  is  the 
minimum  acceptable.  Since  it  is  almost  always  more  costly  to 
build  higher  quality  products,  the  successful  bidder  usually  is 
the  one  who  offers  the  lowest  quality  that  can  just  barely  be 
considered  acceptable.  Thus,  building  high  quality  products  does 
not  necessarily  lead  to  successful  competition. 

Whatever  those  company  goals  may  be  that  impact  R&M  plans,  they 
should  be  written  as  concise  statements  which  will  fit  on  a single 
page  or  briefing  chart,  and  should  include  some  way  of  expressing 
the  relative  importance  of  each  goal.  They  should  be  framed  and 
hung  in  the  project  manager's  office,  and  he  should  present 
these  company  goals  at  the  beginning  of  each  summary  briefing  to 
his  superiors  to  make  sure  they  have  not  changed  their  minds. 

The  goals  should  also  be  frequently  exposed  to  subordinates  and 
associates  as  a reminder. 

More  detailed  objectives  for  the  company  R&M  program  should  then 
be  fashioned  so  as  to  support  those  company  goals,  and  the  project 
manager  should  be  able  to  explain  whey  they  do  support  them. 

Assuming  now  that  the  R&M  objectives  of  the  company  have  been 
identified  and  prioritized,  we  will  turn  to  some  of  the  things 
which  must  be  done  to  delineate  the  best  course  of  action  or 
strategy  to  achieve  those  objectives.  To  develop  that  a strategy, 
the  following  questions  need  to  be  considered  and  answered.  In 
doing  so,  company  policies  must  be  considered  at  the  same  time. 


* Company  Reference  Library 

What  are  the  best  R&M  engineering  and  management 
books  to  have  on  hand?  What  military  standards  and  manuals? 
What  symposia  proceedings  and  technical  journals?  Any  standard 
computer  programs  desirable?  Any  R&M  data  service  subscriptions 
needed?  What  about  parts  selection  references? 
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* Survey  of  Potential  Contract  Requirements 

What  R&M  requirements  should  be  anticipated 
for  the  company’s  product  lines?  What  is  past  history? 

What  changes  to  anticipate  for  future?  How  should  company 
respond  to  warranty  or  incentive  provisions? 

* Facility  Needs 

What  kinds  of  environmental  and  reliability 
test  facilities  should  the  company  own?  What  R&M  service 
subcontractors  are  available?  What  are  data  processing 
needs?  What  about  failure  analysis  facilities? 

* Organization  Alternatives 

What  is  the  best  way  to  organize  so  that  R&M 
engineering  has  the  necessary  influence  in  product  design, 
fabrication,  and  testing?  How  will  the  company  organization 
relate  to  the  Air  Force  R&M  organization? 

* Manpower  Alternatives 

How  many  R&M  engineers,  technicians,  and  ad- 
ministrators are  necessary?  What  experience  and  education 
is  necessary?  What  are  the  educational  opportunities?  What 
is  the  available  service  subcontractor  support? 

* Data  and  Communication  Systems 

What  are  the  requirements  for  collection, 
analysis,  and  distribution  of  data  within  the  company? 

What  will  be  the  Air  Force  contractual  requirements?  What 
summary  data  does  management  need  to  control  the  R&M  pro- 
gram? Does  the  company  need  an  R&M  policy  and  procedure 
guidebook  for  managers  and  engineers? 

Answers  to  the  above  questions  should  result  in  the  selection 
of  the  best  alternatives  to  an  overall  R&M  policy,  organiza- 
tion and  program  for  the  company  independent  of  any  particular 
Air  Force  contract.  More  detailed  "derivative  plans"  will  then 
be  developed  from  those  basic  ground  rules. 

PLANNING  GOVERNED  BY  A PARTICULAR  CONTRACT 

If  the  overall  company  R&M  policies,  facilities  and  organ- 
ization are  in  operation,  "derivative  plans"  for  a particular 
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contract  get  down  to  planning  contract  execution  details  within 
the  framework  of  the  company  R4M  structure.  Both  MIL-STD-785 
for  reliability  and  MIL-STD-470  for  maintainability  specify  the 
preparation  of  program  plans.  These  plans  generally  describe  how 
the  company  plans  to  accomplish  the  contract  tasks  and  deliver 
contract  products.  The  plan  will  show  the  tasks  to  be  accomplished, 
their  time-phasing,  milestones  and  review  times,  the  company  R&M 
organization  and  its  relationship  to  functional  divisions  of 
the  company  (engineering,  manufacturing,  etc.)  the  facilities 
and  subcontracts  to  be  employed  foi  engineering,  failure  analysis 
and  testing,  the  relationship  of  R&M  tasks  to  related  contract 
efforts  such  as  safety,  value  engineering,  and  logistics  studies, 
etc.,  the  data  and  communications  systems  to  be  employed,  among 
other  things.  For  more  specifics,  the  contract  program  plan  task 
would  have  to  be  consulted.  A standard  Data  Item  Description, 

DI-R-3533,  from  the  DoP  Acquisition  Management  Systems  and  Data  { 

Requirements  Control  List,  also  lists  the  specific  contents  required 
in  a Reliability/Maintainability  Program  Plan.  In  addition,  many 
management  factors  pertinent  to  planning  an  R&M  program  have  been 
discussed  in  Chapters  4 and  5. 

In  developing  the  task  time  phasing  and  data  flow,  the  Program 
Evaluation  and  Review  Technique  (PERT)  type  charting  is  extremely 
useful  to  the  planner.  This  job  charting  scheme  is  described  in 
many  management  textbooks.  Summary  level  versions  of  the  PERT 
chart  are  also  instructive  if  included  in  the  program  plan, 
although  the  standards  contain  no  formal  requiremont  to  include 
such  flow  charts. 

ORGANIZING  FOR  R&M 

R&M  engineers  are  normally  placed  in  staff- type  positions 
in  the  company  organization,  and  must  be  given  the  responsibility 
and  authorHy  which  will  allow  them  to  insure  that  company 
products  incorporate  the  R&M  attributes  specified  in  the  contract. 

While  R&M  may  be  a staff  function,  it  is  strongly  recommended 
that  the  responsibility  for  the  R&M  of  equipment  be  assigned  to 
the  design  engineers. 

The  staff  R&M  engineers  must  operate  and  conduct  themselves 
in  such  a way  that  they  will  be  regarded  as  helpful  assistants 
in  accomplishing  detailed  design  and  analysis  tasks,  rather 
than  critical  overlords  to  be  passified.  Consequently,  the 
R&M  staff  engineer  is  often  challenged  by  the  human  relations 


1 


115 


element  in  his  job.  He  has  to  be  helpful  and  persuasive, 
but  firm  and  resolved  when  critical  R&M  issues  are  at 
stake.  He  must  not  only  have  detailed  design  review  re- 
sponsibility, but  must  also  have  authority  to  sidetrack 
designs  which  threaten  R&M  achievement  so  the  project 
manager  or  chief  engineer  can  resolve  the  matter.  It 
is  best,  of  course,  for  the  R&M  staff  engineer  to  work 
out  design  issues  at  the  design  engineer's  level  without 
escalating  the  problem. 

In  large  military  product  companies,  organization  arrange- 
ments similar  to  those  following  are  quite  common. 


PROJECT 

B 

PROJECT 

C 


FUNCTIONAL  SUPPORT 


In  the  Product  Line  Division,  R&M  engineers  are  normally 
located  at  staff  level  in  each  project  office  as  well  as  in 
the  Engineering  Department.  A quality  control  (QC)  group 
will  be  at  staff  level  in  the  Manufacturing  Department. 
Company  policies  and  procedures  must  explain  the  responsi- 
bility, authority  and  working  relationships  between  all 
these  R&M  and  QC  people  at  staff  and  subordinate  levels. 
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There  are  many  variations  on  this  theme,  but  if  R&M  support 
within  the  project  office  and  division  is  much  less  visible 
than  explained  and  depicted  here,  company  achievement  of  Air 
Force  standard  R&M  requirements  may  be  in  jeopardy. 

MANNING  FOR  R&M 

The  task  of  manning  includes  both  the  staffing  of  an  or- 
ganization and  its  training.  We  shall  assume  here  that  the 
Project  Manager  knows  how  to  man  his  project  office  with 
people  trained  in  the  basic  principles  of  R&M  engineering 
that  he  needs,  along  with  the  necessary  levels  of  experience. 

These  R&M  engineers  might  be  hired  "off  the  street"  or  might 
be  assigned  to  the  Project  Office  from  the  Engineering  Depart- 
ment. They  also  might  be  outside  consultants,  at  least  in 
part. 

As  mentioned  before  under  organizing,  a challenging  aspect  of 
manning  an  R&M  activity  is  maintaining  a positive  and  harmoni- 
ous working  relationship  between  the  general  design  engineers 
and  the  R&M  staff  specialists.  (Maintaining  this  good  relation- 
ship is  partly  an  "organizing"  function,  partly  a "manning"  func- 
tion of  proper  hiring,  training  and  orientation,  but  also  a 
"leading"  function  of  motivation).  In  manning  for  R&M  make  sure 
you  hire  people  who  can  work  with  people.  It  is  no  job  for  grin- 
ches,  grouches,  and  negative  thinkers  who  habitually  look  for 
what  is  wrong  and  find  it  painful  to  praise  what  is  right.  It 
also  helps  to  plan  a thorough  orientation  and  training  program 
for  new  people,  not  only  to  explain  procedural  and  organization 
matters,  but  also  to  explain  the  special  human  relations  chal- 
lenges of  the  job.  That  is,  how  to  win  cooperation  and  be  help- 
ful, positive,  timely,  and  persuasive  in  achieving  a good  engi- 
neering response  to  R&M  tasks. 

Informal  contact  between  R&M  specialists  : ' design  engineers 
should  be  frequent.  A formal  design  rev  , for  example,  should 
represent  the  results  of  a series  of  earlier  informal  design  dis- 
cussions between  the  designer  and  the  R&M  people,  thus  represent- 
ing their  best  joint  efforts  rather  than  a "trial  balloon"  created 
by  the  designers  to  be  criticized  by  the  R&M  people.  The  latter 
course  is  a time-wasting,  friction-creating,  and  usually  an  inef- 
fectual procedure. 

Finally,  the  R&M  specialists  should  be  prepared  to  personally 
handle  the  special  statistical  tasks  such  as  R&M  prediction  and 
test  analysis.  They  should  not  badger  the  other  engineers  to 
perform  analyses  which  the  R&M  specialists  are  supposed  to  be 
expert  at  doing  themselves. 
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In  some  companits,  a "Product  Assurance"  group  is  included 
in  the  General  Manager's  staff,  and  that  group's  function 
is  to  oversee  and  control  the  operations  of  R&M,  quality 
control,  safety,  value  engineering,'  etc.,  throughout  all 
departments  and  projects.  This  is  considered  a very  good 
idea. 

The  company  project  office  might  typically  be  organized 
as  follows: 


The  project  manager  is  the  company  counterpart  of  the  Pro- 
gram Manager  in  the  Air  Force  System  Program  Office.  Within 
the  project  office,  the  R&M  staff  group  will  have  engineers 
who  are  specialists  in  avionics,  structures,  engines, 
armament,  etc.  These  specialists  will  guide  and  often  execute 
the  detailed  R&M  tasks  needed  in  those  areas.  The  project 
R&M  group  leader  must  work  in  harmony  with  other  R&M  and  QC 
engineers  working  at  staff  levels  in  Engineering  and  Manu- 
facturing, and  he  must  also  communicate  on  a technical  level 
with  the  R&M  staff  engineer  in  the  Air  Force  System  Program 
Office.  The  Engineering  Data  and  Reports  Group  is  responsible 
ror  the  R&M  data  system  described  in  Chapters  4 and  5,  and  the 
final  composition  of  all  technical  reports  and  specifications. 
Of  course,  they  will  need  inputs  from  the  engineers  in  the 
various  specialty  groups.  Support  for  failure  analysis  and 
environmental  test  work  will  usually  come  from  the  Engineering 
Department  as  needed,  or  possibly  from  subcontractors  admin- 
istered by  the  Engineering  Department. 
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LEADING  FOR  R&M 


Leading  consists  of  making  objectives  clear,  giving  timely 
directions  for  achieving  the  objectives,  (showing  the  way),  and 
motivating  people  so  that  they  will  want  to  achieve  them. 

In  this  function,  the  R&M  staff  supervisor  and  the  project 
manager  must  be  persuasive  communicators.  They  need  to  stress 
the  importance  of  R&M  in  achieving  company  goals  and  in  achieving 
the  specific  objectives  of  the  contract  at  hand.  They  need  to 
make  all  procedural  matters  as  clear  and  unambiguous  as  possible. 
They  need  to  listen  for  complaints  or  evidence  of  friction  between 
designers  and  R&M  specialists  and  need  to  resolve  those  situations 
quickly.  They  should  sponsor  working  group  sessions  between 
designers  and  R&M  specialists 
where  the  R&M  people 
present  helpful  hints 
on  how  to  get  R&M  needs 
folded  into  the  design 
before  the  design  work 
gets  too  far  along. 

They  should  also  explain 
the  military  R&M 
standards  and  manuals, 

(sometimes  a herculean 
task),  and  the 
implications  of 
contractual 
requirements. 
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Finally,  supervisors  should  try  to  come  up  with  some  motivational 
Incentives  or  awards  that  will  encourage  support  and  cooperation 
with  the  R&M  program.  This  is  especially  important  in  the 
quality  control  area  in  the  Manufacturing  Department  when  tedious, 
repetitive  tasks  can  lead  to  boredom  and  apathy. 
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CONTROLLING  FOR  R&M 


Controlling  in  management  is  the  process  of  periodic 
measurement  of  performance  relative  to  establishtd  goals, 
determining  deviations,  and  redirecting  efforts  back  towards 
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the  desired  goals.  The  goals  must  be  visible,  the  performance 
must  be  measurable  and  the  redirection  must  be  timely.  Various 
progress  charting  schemes,  and  design  or  program  reviews  are 
typical  control  methods. 

In  the  R&M  program,  R&M  requirements  provide  the  goals,  and 
RiM  predictions  and  measurements  provide  the  observations 
of  progress.  In  the  quality  control  area,  the  percentage 
of  defective  parts  found  on  incoming  inspection  serves  to 
gauge  acceptable  parts  quality  from  suppliers.  Checkout 
failure  rates  in  production  line  testing  also  serve  as 
measures  of  quality  control  and  reliability  progress.  The 
R&M  data  system  described  in  Chapters  4 and  5 serves  as  the 
data  gathering  tool,  and  the  manager's  summary  of  that  data 
is  his  measure  of  progress  relative  to  the  goals  he  has  de- 
fined at  the  beginning. 

As  mentioned  in  Chapters  4 and  5,  graphical  presentations  of 
R&M  requirements,  predictions,  and  measurements  versus  time 
should  be  maintained  by  the  R&M  staff  office  for  display  and 
discussion  during  program  reviews.  These  charts  should  be 
mounted  on  the  manager's  office  wall  as  a continuing  reminder 
of  where  his  program  has  been  and  where  it  hopes  to  go.  PERT- 
type  charts  are  also  good  Control  tools  as  well  as  planning 
tools,  and  can  be  wall  mounted  for  quick  reference,  easy  up- 
date, and  immediate  visibility. 

Controls  to  be  effective  must  be  timely.  Data  must  be  accur- 
ate and  current,  and  corrective  actions  has  to  be  taken  before 
it  is  "too  late."  That  is,  action  should  be  taken  when  the 
correction  is  relatively  painless  (in  terms  of  pride  and  dol- 
lars), otherwise  the  controlling  action  can  degenerate  to  a 
demotivating,  discouraging  action  which  people  will  try  to 
avoid  through  holding  back  information.  An  example  is  Fail- 
ure Modes  and  Effects  Analysis  (FMEA).  This  is  a valuable 
design  guidance  tool  if  begun  on  time,  but  can  also  become 
merely  a burdensome  activity  if  the  useful  information  comes 
too  late  to  affect  design  and  test  procedures. 

After  design  is  complete,  a very  effective  control  method  to 
insure  that  poor  workmanship,  parts  deficiencies,  and  design 
defects  get  the  attention  they  should,  is  to  focus  management 
attention  on  the  "top  ten"  R&M  problems  of  the  month.  These 
"standout"  deficiencies  can  be  identified  through  statistical 
analysis  of  failures  recorded  in  the  data  system.  Monthly 
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manageinent  review  of  these  R&M  "standouts"  and  their  pro- 
posed solutions  (determined  by  failure  analysis,  and  ap- 
propriate action  by  redesign,  vendor  control,  quality 
control,  etc.)  will  insure  a vigorous  program  for  correc- 
tion of  R&M  deficiencies. 

WARRANTIES 

While  warranties  have  been  used  relatively  little  in 
Air  Force  development  contracting,  they  are  receiving  in- 
creased attention  at  Air  Force  staff  levels  as  one  possible 
approach  to  achieving  enhanced  reliability  of  operational 
equipments.  The  airline  indus^’ry  has  used  equipment  war- 
^ ranties  for  many  years  as  an  incentive  for  suppliers  to 

I provide  reliable  equipment  and  to  give  the  airlines  a more 

i firm  basis  for  planning  operating  budgets,  support  facili- 

ties, and  manpower.  Airline  equipment  warranties  have  been 
widely  used  for  equipments  and  subsystems  such  as  avionics, 
engines,  tires,  batteries,  etc.,  and  generally  apply  for  the 
expected  useful  life  of  the  equipment--perhaps.  for  as  long 
as  ten  years.  The  airlines  have  found  warranties  to  be  cost 
effective  contractual  instruments. 

TYPES  OF  WARRANTIES 

The  World  Airline  Suppliers'  Guide,  published  by  the  Air 
Transport  Association  of  America,  describes  four  basic  types 
of  warranties  used  within  the  industry. 

STANDARD  OR  FAILURE  FREE  WARRANTY 

This  type  of  warranty  ^which  has  been  termed  a "Relia- 
bility Improvement  Warranty"  by  DoD)  applies  to  avionics  as 
well  as  a range  of  other  items.  The  warranty  extends  for  a 
specified  number  of  operating  hours  or  calendar  time  or  a com- 
bination of  both.  The  vendor  normally  assumes  responsibility 
for  labor  and  material  costs  necessary  to  correct  any  failures 
occurring  during  the  warranty  period. 

ULTIMATE  LIFE  WARRANTY 

This  type  of  warranty  applies  to  major  structural  ele- 
ments, such  as  wings,  fuselage,  and  landing  gear.  The  agree- 
ment warrants  that  such  components  will  be  free  from  defects 
for  a stated  number  of  flying  hours. 
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RELIABILITY  GUARANTEE 

The  vendor  is  required  to  have  his  product 
achieve  a stated  mean  time  between  failure  (MTBF). 

Such  agreements  generally  recognize  that  the  initial 
deployment  will  experience  infant  mortality  and  thus 
require  that  the  MTBF  be  demonstrated  after  some  in- 
itial period  of  operational  use.  The  warranty  typi- 
cally runs  until  the  warranted  MTBF  has  been  demon- 
strated for  u stated  number  of  consecutive  months. 

If  at  any  time  the  vendor's  product  fails  to  meet 
the  specified  MTBF,  the  vendor  is  required  to  (a) 
supply  additional  spare  units  to  support  the  air- 
line's operations  until  the  required  MTBF  is  achieved, 
and  (b)  provide  technical  assistance  and/or  modifica- 
tion kits  and  labor  to  achieve  the  warranted  MTBF. 

MAXIMUM  PARTS  COST  GUARANTEE 

Agreements  are  established  with  the  airline 
on  a maximum  materials  cost  per  flying  hour  (or  other 
measure  of  usage)  for  maintaining,  modifying,  repair- 
ing, and  overhauling  selected  items.  Typical  appli- 
cations include  aircraft  tires  and  brakes. 

Other  types  of  warranties,  such  as  maintainability 
guarantees  and  mean  time  to  repair  guarantees,  have 
been  used  on  occasion.  The  types  of  greatest  interest 
in  Air  Force  contracting  are  the  Failure  Free  Warranty 
and  the  Reliability  Guarantee,  or  a combination  of  both. 
The  reason  for  this  is  that  the  greatest  amount  of  sup- 
port costs  are  associated  with  the  kinds  of  equipment 
which  are  suitable  for  these  types  of  warranties;  (avi- 
onics and  ground  electronics).  Warranties  of  these  types 
are  normally  for  a period  of  three  to  five  years. 

USE  OF  WARRANTIES 

Warranties  have  been  used  successfully  by  the  Air 
Force  on  such  things  as  tires,  brakes,  and  hydraulic 
components,  as  well  as  on  electronic  and  electromechan- 
ical equipment.  Criteria  which  define  the  conditions 
under  which  a warranty  would  be  an  appropriate  contrac- 
tual instrument  are  listed  in  Interim  Guidelines  Relia- 
bility Improvement  Warranty  (RIW)  published  in  July  1974 
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by  the  Directorate  of  Procurement  Policy  (AF/LGP).  War- 
ranties are  not  generally  used  when  an  Air  Force  support 
capability  already  exists  or  the  equipment  can  be  read- 
ily repaired  by  the  average  Air  Force  technician  in  the 
field.  The  type  of  equipment  that  is  most  suitable  for 
a warranty  is  new  and  complicated  equipment  which  might 
initially  have  relatively  low  reliability  and  would  re- 
quire a major  effort  to  establish  an  Air  Force  support 
capability. 

The  main  value  of  a warranty  is  that  it  extends  the  con- 
tractor's responsibility  for  his  equipment  for  a long 
period  of  time  beyond  delivery.  As  explained  in  an  ear- 
lier chapter,  a contractor  is  not  normally  motivated  to 
build  any  more  than  the  minimum  acceptable  reliability 
into  his  equipment  since  to  do  so  would  cost  him  more 
and  reduce  his  profit.  The  situation  is  changed  consid- 
erably when  a RIW  is  included  in  the  contract.  Now  he 
must  not  only  consider  production  costs  but  also  support 
costs  for  a long  period  of  time  (3  to  5 years).  He  is 
strongly  motivated  to  build  more  reliability  into  his 
equipment  so  as  to  reduce  his  support  costs  and  make  the 
sum  of  his  production  and  support  costs  a minimum,  and 
thus  his  profit  a maximum.  For  example,  assume  the  con- 
tractor can  make  a reliability  improvement  which  would 
cost  him  $150,000  to  incorporate  and  save  him  $50,000 
per  year  in  repair  costs.  If  the  warranty  period  were 
longer  than  three  years  he  would  obviously  consider  the 
change.  Thus  the  use  of  an  RIW  may  increase  reliability 
and  reduce  the  life  cycle  cost  for  the  Air  Force,  if  the 
warranty  period  is  long  enough  to  permit  the  contractor 
time  to  recoup  any  expenses  he  may  incur  in  improving  his 
product.  Of  course,  this  conclusion  assumes  that  the 
contractor  remains  in  a sound  financial  condition  over 
the  period  of  the  warranty  and  is  able  to  fulfill  its 
provisions. 

A comprehensive  study  of  the  use  of  warranties  for  defense 
avionic  procurement  has  been  made  ("Use  of  Warranties  for 
Defense  Avionic  Procurement,"  ARINC  Research  Corporation, 
RAOC-TR-73-249  dated  Jun  73  (769399).  This  study  recommended 
the  expanded  use  of  failure  free  warranties  since  they  are 
most  easily  administered  and  are  most  compatible  with  existing 
supply  and  maintenance  administration  systems. 

It  is  always  best  for  the  Government  to  state  its  intention 
of  incorporating  warranty  provisions  in  a later  production 
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contract  before  the  equipment  has  been  designed  in  a 
validation  phase  contract.  This  will  allow  the  designer 
to  make  reliability  and  support  cost  trade-offs  which  he 
might  not  otherwise  consider.  It  is  possible,  however, 
to  incorporate  an  RIW  in  the  production  contract  without 
the  intention  being  previously  stated,  although  the  ef- 
fectiveness in  this  case  would  be  less. 


WARRANTY  COSTS 

Airline  experience  with  warranties  indicates  that  the 
cost  of  a warranty  ranges  from  4 per  cent  to  10  per  cent 
of  the  acquisition  cost  per  year  of  coverage.  The  smaller 
figure  is  for  uncomplicated,  inherently  reliable  equipment. 

These  numbers  are  presently  merely  as  an  indication  of  the 
general  cost  of  a warranty.  Each  case  must  be  judged  sep- 
arately. 

SUMMARY 

Warranties  are  widely  and  successfully  used  by  airlines, 
but  have  been  used  comparatively  little  by  the  Air  Force. 

The  Air  Force  is  studying  wider  use  of  warranties.  Criteria 
for  application  of  warranties  are  given  in  Interim  Guidelines 
Reliability  Improvement  Warranty  (RIW)  published  in  July  1974 
by  the  Directorate  of  Procurement  Policy  (AF/LGP).  If  the 
equipment  being  developed  combines  several  of  the  attributes 
listed  there,  the  program  manager  should  surely  ask  his  R&M 
staff  specialist  to  examine  the  use  of  warranty  provisions 
in  the  production  contract.  (However,  such  a decision  should 
be  made  before  the  statement  of  work  for  the  validation  phase 
is  prepared  if  possible,  so  that  potential  contractors  will  be 
aware  of  the  long-te.'m  nature  of  the  program). 

Even  if  a warranty  is  judged  a good  bet  for  the  equipment  pro- 
curement, all  the  R&M  program  elements  described  in  Chapters 
4 and  5 cannot  be  abandoned.  For  example,  if  the  equipment 
is  to  be  eventually  supported  by  Air  Force  logistics,  standard 
military  quality  parts  must  be  used  as  far  as  possible.  Also, 
the  Air  Force  would  not  want  to  be  forced  into  buying  all  kinds 
of  unique  test  equipment  that  a contractor  might  employ  to  sup- 
port warranty  repairs,  if  standard  military  test  equipment  could 
have  been  readily  accommodated  in  the  initial  equipment  design. 

The  combined  warranty  provisions  and  R&M  program  elements  need 
to  be  tailored  to  the  specific  circumstances  and  objectives  of 
the  equipment  being  developed. 
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OTHER  CONTRACT  INCENTIVES 


A reliability  incentive  can  be  incorporated  into 
the  provisions  of  a contract  which  rewards  the  con- 
tractor with  an  added  fee  for  exceeding  a specific 
reliability  and  penalizes  him  for  falling  short.  The 
amount  of  the  reward  and  the  penalty  need  not  be  the 
same,  in  fact  some  schemes  have  rewards  or  penalties 
only.  A warranty  is  also  a form  of  reliability  in- 
centive, since  the  contractor's  profit  or  loss  at  the 
end  of  the  warranty  period  will  depend  largely  on  the 
reliability  attained  by  his  product. 

Here  are  some  examples  of  contractual  reliability  in- 
centives which  have  been  used:  One  manufacturer  who 
was  designing  electronic  equipment  was  rewarded  for 
reducing  the  number  of  parts  in  the  equipment.  The 
idea  here  was  that  fewer  parts  in  the  equipment  would 
make  it  more  reliable.  A satellite  producer  was  re- 
warded for  satellite  lifetime.  The  longer  the  satel- 
lite remained  operational  in  orbit,  the  greater  his 
reward.  A space  launch  vehicle  manufacturer  was  in- 
centivized  on  launch  success.  He  received  a reward 
each  time  there  was  a successful  launch  and  was  pen- 
alized for  each  failure.  An  electronic  equipment  man- 
ufacturer was  given  a reward  or  penalty  based  on  the 
number  of  failures  that  occurred  during  the  reliability 
demonstration  test.  Other  such  schemes  have  also  been 
used. 

Reliability  incentives  have  not  been  very  widely  used, 
although  Air  Force  policy  encourages  the  use  of  in- 
centives in  general.  Perhaps  the  reason  is  that  it  is 
difficult  to  structure  an  incentive  around  objective 
R&M  performance  criteria,  free  from  things  beyond  the 
contractor's  control,  so  that  financial  settlements 
can  be  readily  agreed  upon  after  hardware  delivery. 

Another  reason  may  be  that  there  are  many  people  who 
are  not  personally  convinced  that  incentives  have  the 
positive  value  often  touted.  They  believe  that  the 
contractor  is  already  well  paid  under  the  contract  to 
produce  equipment  with  the  required  performance,  that 
this  is  the  basic  incentive,  and  to  pay  an  added  in- 
centive is  a waste  of  money. 

Another  incentive  for  a contractor  to  build  reliable  equip- 
ment is  the  company  reputation.  A company  desires  a good 
reputation  for  several  reasons,  not  the  least  of  which  is 
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its  effect  on  subsequent  business  and  profits.  In  fact, 
a company's  very  survival  may  depend  on  the  quality  of 
its  products.  The  value  of  a good  reputation  is,  of 
course,  not  measurable  and  its  importance  to  different 
companies  varies.  It  should  be  noted  that  reputation 
is  less  important  to  a company  in  its  military  business 
than  in  its  commercial  business,  because  of  the  strict 
rules  that  military  source  selection  proceedings  must 
follow. 

The  program  or  project  manager  should  ask  his  staff  R&M 
engineer  and  contract  people  to  consider  the  use  of  spe- 
cial contract  incentives  to  enhance  the  chances  of  reach- 
ing R&M  objectives.  The  DoD/NASA  Incentive  Contracting 
Guide,  AFP  70-1-5,  gives  guidelines  for  developing  in- 
centive provisions. 

As  with  warranties,  all  the  R&M  program  elements  discussed 
in  Chapters  4 and  5 cannot  be  abandoned  and  replaced  by 
incentive  provisions  in  the  contract.  That  would  have 
little  chance  of  success. 

The  overall  R&M  program,  including  contract  incentive 
provisions,  needs  to  be  tailored  to  the  specific  equip- 
ment type  and  program  objectives  by  the  R&M  staff  spe- 
cialists in  the  program  or  project  office. 
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Chapter  7 

RELIABILITY  AND  MAINTAINABILITY 
OF  COMPUTER  SOFTWARE 


I 


INTRODUCTION 

The  two  earlier 
chapters  on  hardware 
reliability  and  main- 
tainability start  off 
with  a picture  window 
containing  the  mili- 
tary standard  which 
governs  the  content 
of  each  chapter.  In 
the  area  of  computer 
software  R&M,  we  have 
no  military  standard 
and  therefore  our  win- 
dow is  empty.  This 
lack  of  a standard  is 
partly  due  to  the  rel- 
ative newness  of  com- 
puter software  technology,  although  software  has  been  a 
major  part  of  some  Air  Force  systems  for  more  than  20 
years,  beginning  with  the  early  missile  and  air  defense 
systems.  Whatever  the  reasons  may  be  for  not  having 
software  R&M  standards.  Government  experiences  with  large 
systems  over  the  past  10  years  (Apollo,  Safeguard,  defense 
satellite  system,  etc.),  and  the  mushrooming  new  uses  of 
digital  systems,  have  all  shown  that  the  software  community 
needs  to  focus  attention  on  R&M.  Indeed,  response  to  this 
need  has  been  reflected  in  technical  literature  and  in  new 
Air  Force  R&D  work  in  recent  years. 

Software  in  Air  Force  systems  has  been  growing  like  a snow- 
ball rolling  downhill.  This  has  resulted  from  production 
of  microscopic  digital  circuit  devices  that  combine  increased 
reliability,  lower  unit  cost,  and  lower  power  requirements. 
These  smaller,  better  devices  have  inspired  many  new  appli- 
cations which  have  in  turn  spawned  the  need  for  more  and  more 
software.  The  trade  literature  has  claimed  that  the  cost  of 
new  software  ranges  from  two  to  ten  times  the  cost  of  hardware 
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on  which  it  operates,  furthermore,  estimates  show  that 
the  Air  Force  now  spends  about  a half  billion  dollars 
annually  on  its  related  software.  Software  is  biq 
business  and  is  a major  cost  and  risk  segment  of  most 

new  systems. 
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While  we  do  not  have  any  software  standards  whch  focus  on 
R&M,  other  general  regulations  do  have 
For  example,  we  have  the  300-senes  (Data  Automation) 
regulations  and  manuals  which  for  many  years 
procurement  and  management  of  data  processing 
support  applications.  This  series,  in  varying  degrees, 
has  also  governed  research  and  development 
In  addition,  various  configuration  management  procedures 
have  been  applied  to  software  development.  Of  course, 
those  regulations  and  standards  have  benefitted  R&M  some- 
what. 
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A series  of  standards  and  guidebooks,  although  not  specifically 
dealing  with  software  R4M,  can  provide  guidance  on  the  require- 
ments for,  and  possible  contents  of,  a software  R&M  program. 
Some  of  the  more  recommended  ones  are: 

AFSCP  8(Ji9-7,  "Configuration  Management,"  Dec  77 
(specifically.  Chapter  6 entitled  Configuration 
Management  of  Computer  Programs). 

AFSCR  74-1,  "Quality  Assurance  Program,"  Nov  78 

MIL-S-52779,  "Software  Quality  Assurance  Program 
Requirements 

ESD-TR-77-225,  "Software  Acquisition  Management 
Guidebook:  Software  Quality  Assurance,"  Aug  77 

ASD-TR-78-8,  "Airborne  System  Software  Acquisi- 
tion Engineering  Guidebook  for  Quality  Assurance," 

Aug  77 

The  above  regulations  and  guides  were  developed  around  the 
framework  of  AFR  800-14,  Sep  75,  a two  volume  set  of 
regulations  dealing  specifically  with  the  Air  Force  policies 
and  responsibilities  for  the  management  of  software 
acquisitions  for  systems,  as  defined  in  AFR  800-2,  "Acquisi- 
tion Program  Management,"  Nov  77,  throughout  the  software 
life  cycle  (both  development  and  operational  useage). 

Volume  I of  AFR  800-14,  "Management  of  Computer  Resources  in 
Systems,"  establishes  the  management  responsibilities  for 
software  acquisitions  among  the  various  user,  developing,  and 
support  commands.  Volume  II  of  AFR  800-14,  "Acquisition  and 
Support  Procedures  for  Computer  Resources  in  Systems," 
specifically  spells  out  the  various  software  life  cycle  events, 
planning  documents,  and  management  principles  that  must  be 
rigidly  followed  in  acquiring  software  for  systems  in 
accordance  with  AFR  800-2.  In  essence,  then,  AFR  800-14 
provides  the  outline  for  the  nnethods  the  Air  Force  at  1 its 
contractors  will  use  in  acquiring  software.  The  office  of 
primary  responsibility  for  AFR  800-14  is  AFSC/XRF. 

Before  discussing  the  separate  engineering  elements  of  soft- 
ware development,  let  us  review  several  terms.  The  meaning 
of  "software"  is  well  known.  It  is  the  set  of  written 
instructions  which  tell  a digital  computer  what  to  do.  Of 
course,  these  instructions  have  to  be  written  in  a simple 
and  precise  language  which  the  computer  can  understand. 
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Usually,  the  instructions  are  first  written  in  a "higher 
level"  language  such  as  FORTRAN  IV,  which  is  easy  for 
people  to  work  with.  This  set  of  instructions  is  called  the 
"source  program."  The  program  is  then  translated  via 
another  computer  program  (called  a "compiler')  into  machine 
language  (the  "object  program")  which  is  efficient  for  the 
system  computer  to  read  and  execute. 

Throughout  this  process  of  software  development,  many  kinds 
of  mistakes  can  be  made  which  lead  to  system  reliability 
problems.  These  mistakes  can  be  grouped  as  follows: 


* Problem  definition 

In  the  system  engineer's  conception  of 
the  overall  operational  system  problem 
to  be  solved,  some  task  or  situation 
has  been  misinterpreted  or  has  been  over- 
looked and  left  out  of  the  problem  form- 
ulation. Sometimes,  as  the  definition 
of  the  problem  proceec*s  into  greater  de- 
tail, interface  and  timing  errors  arise. 

* Computation  logic 

The  computer  program  designer  has  made 
logical  mistakes  or  misinterpretations 
in  converting  the  problem  statement  in- 
to computational  steps  or  unanticipated 
combinations  of  circumstances  cause  er- 
rors in  execution. 

* Grammar 

The  programmer  or  the  program  translator 
(compiler  program)  have  made  mistakes  in 
applying  the  grammatical  rules  of  the  pro- 
gramming languages. 

* Typographical 

The  keypunch  operator,  the  keypunch 
machine,  the  card  reader,  tape  unit, 
or  other  data  transducers  have  made 
mistakes  in  converting  symbols  from 
one  form  to  another. 
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Editing,  debugging  and  system  testing  routines  are  designed 
to  flush  out  these  mistakes.  The  first  category  (problem 
definition)  contains  the  kind  of  mistakes  which  are  the 
most  difficult  to  uncover  and  cause  the  most  operational 
heartburn,  since  they  might  never  be  discovered  until 
revealed  by  system  failure  during  some  unusual  mission. 

By  that  time,  computer  program  changes  are  very  costly  and 
disruptive,  and  of  course  mission  failure  could  mean  a 
disaster.  This  difficulty  also  applies  in  some  degree  to 
the  second  category  (computation  logic),  since  large  computer 
programs  may  contain  millions  of  possible  logical  flow 
paths  with  some  never  exercised  during  tests.  Some  paths 
may  come  into  use  only  by  chance,  years  after  deployment. 

The  term  "software  reliability"  refers  to  the  degree  that 
a software  package  can  be  expected  to  perform  its  intended 
function  according  to  expected  mission  profiles.  While 
several  numerical  measures  of  software  reliability  have  been 
proposed  and  used  in  some  instances,  no  generally  accepted 
measure  of  software  reliability  has  been  adopted  for  wide- 
spread useage. 


The  term  "Software  maintainability"  refers  to  the  degree  a 
software  product  facilitates  updating  to  satisfy  new  require- 
ments  or  modification  to  correct  mistakes.  Again,  no  single 
method  for  measuring  software  maintainability  has  been 
adopted,  even  though  the  major  portion  (up  to  70%)  of  soft- 
ware life  cycle  costs  are  for  software  maintenance. 


The  Inform! on  Sciences  Division  (IS)  of  Rome  Air  Development 
Center  (RADC)  is  the  focal  point  within  the  Air  Force  for^ 
software  reliability  expertise.  An  extensive  program  for 
establishing  tools  and  procedures  for  improving  software 
reliability  has  been  underway  for  several  years.  This  program 
has  resulted  in  formulation  of  RADC  Specification  CP 
0787796100,  "Computer  Software  Development  Specification," 

Mar  78,  which  is  available  for  guidance  and  information.  Of 
more  importance  to  this  manual  is  the  forthcoming  Oct  79  RADC 
handbook  on  software  reliability  prediction  which  will  describe 
in  detail  the  use  of  current  methodologies  for  predictir.j 
software  reliability.  This  handbook  is  intended  as  an  interim 
product  that  will  eventually  become  the  software  counterpart 
to  MIL-HDBK-217,  "Reliability  Prediction  of  Electronic 
Equipment". 
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ELEMENTS  OF  SOFTWARE  DEVELOPMENT 


The  following  diagram  shows  the  major  elements  of 
a software  development  cycle.  Note  that  this  is  not 
a software  R&M  program,  but  a system  software  develop- 
ment program  in  general.  Software  RS’M  has  simply  not 
developed  to  the  point  where  it  can  be  discussed  as  a 
separate  engineering  discipline.  However,  each  of  the 
steps  is  described  later  emphasizing  R&M  considerations. 
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SYSTEM  SOFTWARE  DEVELOPMENT  CYCLE 


The  diagram  introduces  some  more  terms  that  need  expla- 
nation. The  term  "system"  refers  to  the  combined  soft- 
ware/hardware system  including  the  computer  and  every- 
thing that  functions  with  it  to  provide  and  handle  numer- 
ical data.  The  software  "package"  is  the  complete  set  of 
computer  programs  which  govern  the  processing  of  system 
data.  This  package  is  organized  into  separate  "programs,' 
"subprograms,"  "subroutines,"  etc.,  with  the  smallest 
functional  subdivision  being  a "module"  or  "unit." 
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It  is  desirable  for  this  unit  to  be  small  enough  to  fit  on 
one  page  of  source  code  instructions,  and  junior  programmers 
are  assigned  the  job  of  designing,  coding  and  debugging  one 
or  more  of  these  units  in  a large  system  problem.  Senior 
programmers  or  system  analysts  are  responsible  for  organizing 
and  interfacing  the  larger  groupings  of  units  (subprograms, 
subroutines,  programs,  etc.;,  and  those  programmers  must  be 
sure  the  code  is  written  so  that  all  units  operate  together 
in  harmony. 
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The  "subroutines"  sitting  on  the  sidelines  are  standard  sub- 
programs which  the  programmer  can  call  into  use  at  any  level 
to  carry  out  routine  calculations  such  as  square  toot,  stand- 
ard deviation,  sine  function,  etc.  While  the  functional 
organization  of  the  software  package  is  charted  from  large 
subdivisions  on  the  top  to  the  smallest  units  on  the  bottom, 
the  computational  sequence  might  follow  almost  any  path 
through  the  chart.  Therefore,  communications  between 
programmers  and  documentation  of  coding  rules  and  decisions 
must  be  highly  disciplined. 
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After  this  generalized  view  of  the  software  life  cycle, 
the  following  diagram  depicts  the  software  life  cycle  as 
detailed  in  AFR  8(50-14.  Brief  discussions  of  each  of 
these  phases  is  given  below,  with  a more  comprehensive 
discussion  provided  in  Chapter  2 of  AFR  800-14,  Vol . II. 

The  analysis  phase  is  the  period  of  time  where  the 
functional  performance  requirements  of  a software  system 
are  defined.  This  phase  normally  begins  after  release  of 
the  system  specifications  and  terminates  with  completion 
of  the  Preliminary  Design  Review  (PDR).  The  purpose  of  a 
PDR  is  to  evaluate  the  progress,  consistency,  and  technical 
adequacy  of  a selected  design  and  test  approach,  as  well  as 
establish  stability  with  program  requirements.  During  the 
analysis  phase  the  selection  of  the  design  approach  to  be 
used  among  the  various  alternatives  is  completed  and  the 
computer  program  design  specification  is  produced. 

In  the  design  phase  the  design  approach  decided  upon  in  the 
analysis  phase  is  developed  to  include  mathematical  models, 
functional  flow  charts  and  detailed  flow  charts.  Detailed 
flow  charts  are  used  to  define  the  information  processing 
in  terms  of  logical  flow  and  operations  to  be  performed  by 
the  computer  programs.  The  relationship  between  computer 
programs,  and  the  interfaces  between  the  software  and  hard- 
ware system  components  are  also  defined  at  this  time.  This 
phase  culminates  in  a Critical  Design  Review  (CDR),  which  is 
a formal  review  to  establish  design  integrity  at  the  detailed 
flow  chart  level  prior  to  actual  coding  and  testing.  The 
design  phase  also  culminates  in  a preliminary  computer  pro- 
gram product  specification. 

The  coding  and  checkout  phase  is  where  the  detailed  design  is 
. translated  into  actual  program  code  and  the  initial  testing 
of  the  code  is  performed,  usually  by  the  programmer  of  that 
code.  This  initial  testing  normally  is  designed  to  check  for 
correct  outputs  using  predefined  inputs.  Successful  comple- 
tion of  this  testing,  often  referred  to  as  "unit"  testing, 
leads  into  the  test  and  integration  phase. 

In  the  test  and  integration  phase,  the  various  computer  pro- 
grams are  tested  against  the  requirements  as  stated  in  the 
computer  program  development  specification.  The  testing 
process  begins  with  testing  of  individual  computer  programs 
and  progresses  to  integration  of  1)  the  computer  programs 
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into  a total  software  package,  and  2)  the  software  and  hard- 
ware components  into  a total  system.  This  phase  is  concluded 
with  a formal  qualification  of  the  software  produced  and 
development  of  the  completed  computer  program  product  speci- 
fication. 

The  installation  phase  includes  the  loading  and  running  of 
the  computer  programs  after  successful  completion  of  integra- 
tion testing  and  formal  qualification.  During  this  phase  the 
system  is  checked  to  insure  its  performance  is  within 
specified  levels  of  confidence.  The  operational  and  support 
phase  covers  the  time  when  the  operational  suitability  of  the 
system  is  assessed  and  the  capability  of  the  system  to 
operate  in  an  operational  environment  is  evaluated.  Support 
phases  cover  all  the  activities  and  resources  required  to 
insure  that  the  software  continues  to  meets  its  required 
operational  capabilities.  Included  in  this  phase  are  the 
activities  of  software  maintenance  performed  to  either  1) 
correct  software  errors  that  were  not  detected  during  testing, 
2)  make  changes  in  the  software  to  maintain  its  operational 
capabilities  at  current  levels,  and  3)  make  changes  in  the 
software  to  upgrade  or  modify  the  operational  capabilities  of 
the  system.  Activities  during  this  phase  also  include  making 
changes  in  the  supportive  documentation,  such  as  program 
listings  and  the  product  specification,  to  accompany  any 
changes  in  the  code  or  in  the  requirements.  This  phase  is 
the  largest  in  terms  of  life  cycle  costs,  with  an  estimated 
60-70%  of  the  total  software  life  cycle  cost  being  accounted 
for  during  this  phase. 

The  following  chart  lists  the  principal  elements  of  a program 
to  develop  the  software  package  for  a large  system.  It  also 
shows  the  phasings  and  relative  importance  judged  reasonable 
for  an  "average"  development  program.  The  percentage 
distribution  of  contractor  manhour  effort  gives  some  idea  of 
what  to  expect,  and  can  vary  widely  from  one  program  to 
another  depending  on  circumstances.  A particular  program 
would  have  to  be  planned  by  experienced  system  software 
people  to  fit  those  circumstances. 

Notice  how  early  the  requirements  and  system  analysis  are 
expected  to  be  complete— early  in  the  validation  phase. 

That's  because  software  design  cannot  proceed  very  far  until 
certain  mission  decisions  are  made.  These  include  exact 
mission  problems  to  be  solved,  data  inputs  and  outputs,  com- 
puter design,  prograttming  languages,  etc.  Any  basic  hardware 
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ELEMENTS  OF  SOFTWARE  DEVELOPMENT 


Element 


Requirements  Definition 
System  Analysis 
Package  Design 
Unit  Design.  Code  & Debug  ^5 
.Package  Integration  & Test 
System  Integration  & Test  30 
Acceptance  Test 
Program  Plan 
Specifications 
Data  System 
Program  Review 
Test  Plan 
Technical  Manuals 


1 Life  Cycle  Phase 

Conceoutal 

Validation 

Full  Scale 
Development 

Production 

Deployment 

xxxxxxxxxx 
xxxxxxxxxx 
oocjooooooooooo 
xxxxxxxxxx 
— ooooxxx 


changes  which  come  later  can  cause  big  pieces  of  software 
effort  to  be  scrapped  leading  to  cost  overruns  and  schedule 
crises.  The  software-related  hardware  and  mission  parameters 
have  to  be  identified  and  pinned  down  early. 

Notice  also  that  the  software  design,  code  and  test  work  trail 
off  into  the  deployment  phase.  While  not  a desirable  circum- 
stance, it  is  almost  inevitable  with  large  software  systems 
because  of  their  size,  complexity  and  the  relative  ease  with 
which  changes  can  be  made.  Planners  must  recognize  this 
reality  and  set  up  a documentation  and  approval  system  which 
will  keep  the  changes  under  control. 

Definition  of  Requirements 

Software  requirements  define  the  overall  mission  problem 
to  be  solved  by  the  software,  the  operational  constraints, 
and  any  fixed  interfaces  with  system  hardware  and 
people.  Requirements  must  cover  the  following  kinds  of 
information: 

* Mission  problems  to  be  solved  by  the  soft- 
ware system 

* Software-related  system  hardware  design 
decisions  not  subject  to  tradeoff  studies 

* Software  design  contra ints  imposed  on  Air 
Force  systems 

* Input  data  sources,  rates  and  formats  (if 
established) 

* Output  data  destinations,  rates,  and  for- 
mats (if  established) 

* Adaptability  required  for  system  modifications 
in  operational  use 

* Software-dependent  maintenance  concepts  and 
plans 

* Security  needs 

* Operational  hazards  and  environment 

* Reliability  and  maintainability  needs 

Requirements  are  determined,  so  far  as  possible,  by  the  System 
Program  Program  Office  as  an  in-house  task.  The  work  is  done 
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before  the  first  contract,  depending  heavily  upon  inputs 
from  the  using  coitmand  and  AFLC.  After  the  first  contract, 
the  contractor  will  further  refine  and  define  the 
requirements  through  systems  analysis  and  discussions  with 
the  System  Program  Office,  As  mentioned  earlier,  require- 
ments must  be  pinned  down  by  the  early  part  of  the 
validation  phase,  otherwise  subsequent  software  effort  will 
have  to  proceed  on  the  basis  of  assumptions.  This  can 
cause  a lot  of  waste,  if  the  assumptions  do  not  come  true 
later  on.  Requirements  are  documented  in  the  program  plan, 
system  specifications  and  interface  specifications. 

System  Analysis 

System  analysis  proceeds  in  parallel  with  requirements 
definition,  and  evaluates  the  system  design  tradeoffs 
between  hardware  and  software.  It  considers  computer  hard- 
ware options,  maintenance  options,  and  in  general,  all  of 
the  software-related  hardware  alternatives.  The  objective 
is  to  design  the  hardware/software  system  so  as  to  maximize 
the  chances  of  success  at  the  lowest  life  cycle  cost.  These 
chosen  design  options  are  documented  in  system  and  interface 
specifications  used  by  the  software  designers.  The  first 
set  of  A's  on  the  chart  refers  to  delivery  of  these  hardware 
parameters  to  the  specification  writers. 

Another  important  area  of  system  analysis  which  continues 
through  the  middle  of  full  scale  development,  is  the  develop- 
ment of  schemes  for  system  testing  and  acceptance.  The 
thoroughness  of  these  schemes  directly  affects  the  verification 
of  software  R&M.  Test  schemes  are  documented  in  the  system 
test  plans  and  acceptance  specifications.  The  second  set  of 
A's  on  the  chart  refers  to  delivery  of  this  test  planning 
information  to  the  test  plan  writers. 

Package  Design 

Package  design  refers  to  the  development  of  the  complete 
software  system  functional  organization.  That  is,  the  over- 
all task  of  the  software  system  is  broken  down  into  functional 
categories  and  subcategories  all  the  way  to  the  unit  level. 

(The  process  is  analogous  to  organizing  a large  group  of 
people  with  diverse  skills  to  carry  out  a project).  To 
enhance  R&M  of  a large  software  system,  this  software  functional 
organization  must  be  thorough,  well  documented,  and  all 
interface  rules  between  functional  elements  must  be  precisely 
defined  and  their  application  carefully  controlled. 
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A "chief  programmer"  or  a senior  software  system  engineer 
must  be  assigned  to  oversee  and  manage  this  whole  process. 
Subordinate  programmers  responsible  for  the  separate  programs 
in  the  functional  categories  will  be  assigned  to  him. 

In  other  words,  there  will  be  a hierarchical  organization  of 
people  (programmers)  with  supervisors  and  subordinates  that 
pretty  much  parallels  the  functional  organization  of  the 
software  system.  The  chief  programmer  must  not  only  be  an 
engineer  experienced  in  development  of  large  software 
systems,  but  must  also  be  skilled  in  applying  the  traditional 
management  tools  to  plan,  organize,  man,  lead,  and  control 
his  people  and  project. 


THE  PROGRAMMER’S  BUREAUCRACY 


In  turn,  the  subordinate  manager  of  each  program  or  sub- 
program will  plan,  organize,  lead  and  control  the  detailed 
coding,  testing,  and  documentation  of  programming  within 
his  domain  using  the  ground  rules  laid  down  by  the  chief 
programmer.  At  the  same  time,  each  subordinte  manager 
will  devise  schemes  for  testing  to  insure  quality.  The 
results  of  this  work  are  documented  in  the  test  plan, 
data  system  and  specifications  discussed  below. 

In  addition  to  organizing  the  whole  operation,  the  chief 
programmer  must  identify  the  source  program  langauges  to 
be  used  (from  system  analyses  documented  in  the  system  and 


interface  specifications)  and  the  general  rules  for  program 
structure  and  progress  documentation  throughout  his 
organization.  The  programming  rules  should  be  documented  in 
one  of  the  computer  program  design  specifications. 


To  enhance  the  readability  and  testability  of  the  computer 
programs,  "structured  programming"  techniques  should  be 
employed.  In  part,  this  means  that  the  programmer  is  restricted 
to  a small  set  of  standard  language  constructs  which  prevent 
him  from  skipping  to  some  remote  segment  of  the  computational 
sequence.  Unbridled  use  of  "go  to"  instructions  can  lead  to  a 
maze  of  possible  computational  flow  paths  making  the  program 
very  difficult  to  comprehend,  document  and  check  out,  Further- 
more,  the  possiblity  of  logical  traps  or  dead  ends  is  greatly 
enhanced.  Therefore,  a "structured  programming"  approach 
should  be  followed.  Software  specialists  will  know  what 
structured  programming  means. 

Unit  Design,  Code  and  Debug 

Another  attribute  of  "structured  programming"  is  the  size 
restriction  on  program  units  or  modules.  The  unit  is  typically 
defined  to  be  about  50  lines  of  program  code  which  will  fit  on 
one  listing  page.  Furthermore,  the  unit  will  have  only  one 
link  from  the  preceding  unit  and  one  link  to  the  following  unit. 
These  rules  enhance  readability,  comprehension,  and  independent* 
testability  of  each  unit.  Each  "Chief"  will  supervise  the 
design,  code,  debug,  and  test  of  his  group's  output.  He  may, 
of  course,  be  responsible  for  a number  of  units  in  the  overall 
software  program.  He  will  document  his  work  in  the  data 
system  and  the  appropriate  computer  subprogram  design 
document  noted  below  in  the  discussion  of  specifications. 

Package  Integration  and  Test 


Package  integration  and  test  means  that  units,  subprograms, 
subroutines,  etc.,  and  programs  are  assembled  and  tested  in 
groups  of  increasing  size  until  the  entire  software  package  is 
put  together.  This  assembly  and  testing  is  usually  done  with 
the  aid  of  general  purpose  computers,  since  the  operational 
hardware  computer  may  not  be  available  until  late  in  full 
scale  development.  The  Test  Plan  is  used  throughout  this 
process,  and  results  are  documented  in  the  data  system.  The 
thoroughness  of  this  element  of  the  software  development  process 
IS  critical  to  software  reliability. 
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System  Integration  and  Test 

System  integration  and  test  means  that  the  software 
package  is  inserted  into  the  operational  hardware,  and 
complete  system  tests  are  run  to  insure  that  hardware  and 
software  are  compatible  and  that  operational  requirements 
can  be  fulfilled.  This  element  is  also  critical  to 
verification  of  operational  suitability.  It  occurs  in  the 
final  phases  of  full  scale  development,  and  hopefully, 
only  minor  changes  will  be  necessary  then.  The  Test  Plan 
is  used  to  conduct  these  tests. 

Acceptance  Test 

The  software  acceptance  test  is  defined  in  the  Test 
Plan  and  possibly  in  an  overall  system  acceptance  spec- 
ification. This  test  is  the  final  test  which  formally 
establishes  acceptability  of  software  products  for 
delivery  under  the  development  or  production  contract. 

Preparation  of  numerical  acceptance  criteria  is  hampered  by 
the  lack  of  any  widely  accepted  measures  of  software  R&M. 
Nevertheless,  the  Air  Force  Program  Office  must  be  sure 
that  acceptance  criteria  are  developed  during  the  conceptual 
and  validation  phases.  This  is  partly  an  in-house  task 
f using  help  from  Government  software  engineers,  but  is  also 

a task  for  the  contractors  under  system  analysis  and  package 
and  unit  design.  Criteria  are  documented  in  the  Test  Plan 
and  acceptance  specifications. 

Program  Plan 

The  Program  Plan  outlines  and  explains  all  elements  of 
the  software  development  effort.  It  shows  requirements, 
interfaces,  organization,  task  breakdown,  responsibilities, 
schedules,  and  the  approach  to  solving  all  the  software 
development  problems  so  as  to  fulfill  the  requirements  on 
schedule  and  within  projected  cost.  This  plan  is  developed 
mostly  by  the  contractors  during  conceptual  and  early  vali- 
dation phases,  but  must  be  continuously  updated. 

For  program  planning  purposes,  several  documents  explaining 
all  elements  of  the  software  development  effort  are  prepared 
in  accordance  with  AFR  8|30-14.  The  major  documents  are  the 
program  management  directive  (PMD),  program  management  plan 
(PMP),  computer  resources  integrated  support  plan  (CRISP), 
and  the  computer  program  development  plan  (CROP).  The  PMD 
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is  used  by  HQ  USAF  to  identify  the  technical  and  managerial 
expertise  responsible  to  the  program  manager  (PM)  for  manag- 
ing the  acquisition  of  computer  resources  for  systems  being 
acquired  per  AFR  800-2.  The  PMD  focuses  on  both  the 
computer  program  development  and  the  total  system  integration. 
The  PMP  is  prepared  by  AFSC  in  conjunction  with  AFLC  and  the 
using  command.  Its  purpose  is  to  document  the  complete  plan- 
ning for  the  acquisition  management  of  the  computer  resources, 
including  computer  program  requirements,  major  project 
milestones,  identification  of  required  total  system  interfaces 
and  configuration  management  concepts  to  be  used  on  that 
particular  project. 

The  CRISP  identifies  the  organizational  relationships  and 
responsibilities  for  the  management  and  technical  support  of 
computer  resources.  The  CRISP  is  used  during  the  develop- 
ment phases  to  identify  the  computer  resources  necessary  to 
support  the  system  after  management  responsibility  has  been 
transferred  from  AFSC  to  AFLC  and  the  using  command.  During 
the  preparation  of  the  CRISP  the  Computer  Resource  Working 
Group  (CRWG)  is  formed,  with  members  from  AFSC,  AFLC  and  the 
' using  command.  The  CRWG  is  responsible  for  preparing  and 

1 updating  the  CRISP  and  insuring  its  proper  implementation  in 

the  program  management  responsibility  transfer  (PMRT)  plan. 

The  CPDP  is  developed  generally  by  the  contractor  and  is  used 
to  identify  those  activities  needed  to  develop  and  deliver  a 
computer  software  package,  including  all  necessary  support 
resources.  The  CPDP  addresses  such  items  as  the  development 
schedule,  procedures  for  monitoring  development  status,  the 
approach  to  developing  all  necessary  documentation,  and  any 
required  engineering  practices,  such  as  the  use  of  structured 
coding. 

[ Specifications 

^ Specifications  formally  and  precisely  document  all  require- 

[ ments  and  design  decisions.  They  may  be  grouped  into  several 

I categories: 

* System  Specification 

Defines  the  system  requirements  and  the 
overall  hardware/software  system  design 
in  top  level  detail. 
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Software  Performance  Specification 

Defines  the  software  requirements,  soft- 
ware design  ground  rules,  selected  soft- 
ware-dependent hardware  parameters, 
interface  identification,  and  overall 
structure  of  the  software  system.  This 
specification  goes  into  a second  level  of 
detail  below  the  System  Specification, 

Interface  Specifications 

Defines  the  interface  design  details 
between  software  and  hardware  elements  and 
between  software  subdivisions.  It  goes  into 
a second  level  of  detail  below  the  preceding 
Software  Performance  Specification. 

Software  Design  Specification 

Defines  and  describes  the  computer  programs 
that  will  meet  the  Software  Performance 
Specifications  in  functional  flow  diagram 
detail.  It  also  defines  the  progranming 
scheme  and  rules  which  will  be  used  by  pro- 
grammers to  implement  the  functional  elements 
in  computer  code. 

Subprogram  Design  Document 

Gives  a detailed  technical  description  of 
each  subprogram  including  input,  output, 
functional  flow,  narrative  description, 
limitations,  interfaces,  and  mathematical 
equations  solved  or  operations  performed. 

It  also  describes  the  tests  used  to  check 
it  out. 

Common  Data  Base  Design  Document 

Gives  a detailed  technical  description  of 
all  data  items  used  by  the  software  system. 
This  includes  constants,  variables,  and 
tables.  Details  include  data  name,  table 
index,  purpose,  dimensions,  units,  initial 
values,  range  of  values,  exact  format,  etc. 

Acceptance  Specification 

Defines  the  criteria  to  be  used  in  judging 
formal  acceptability  of  software  products 
under  contract. 
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Data  System 


The  data  system  also  called  the  program  support  library, 
is  designed  to  provide  management  control  information  and 
documentation  discipline.  It  will  consist  of  some  kind  of 
periodic  reporting  procedure  where  every  programmer  will  be 
required  to  submit  at  least  a weekly  report  on  his  effort. 

The  reports  might  include  estimates  of  coding  completion  on 
assigned  units,  numbers  and  classifications  of  errors  found 
in  debugging  and  testing,  information  shortages  which  hamper 
coding  progress,  specification  errors  discovered,  manhours 
spent  on  separate  units,  documentation  contributions,  etc. 
Listings  of  each  run  are  also  collected  and  stored  in  this 
system.  The  chief  programmer  will  have  an  administrative 
staff  to  compile  the  reports  into  composite  surtmary  charts, 
graphs  and  narratives  for  use  in  management  reviews.  The 
data  system  must  also  cover  status  of  the  documentation, 
and  some  very  disciplined  scheme  must  be  devised  to  Insure 
that  documentation  keeps  up  with  changes  in  requirements, 
system  design  and  software  design. 

Notice  in  the  chart  of  software  development  elements  presented 
above  that  the  data  system  continues  through  production  and 
deployment.  This  means  that  the  Air  Force  must  adopt  a data 
system  for  use  throughout  the  software  life  cycle.  In  contrast 
to  hardware,  software  is  relatively  easy  to  change  in  the 
field  and  documentation  changes  must  be  thoroughly  discipl ir'ied. 

Program  Review 

The  contractor  will  have  frequent  in-house  program  reviews, 
and  the  Air  Force  less  frequent  reviews.  In  the  Air  Force 
program  reviews,  overall  program  progress  is  reviewed  and 
compared  with  the  CPDP.  Also,  a technical  review  of  the 
software  is  performed  by  the  Program  Office  backed  up  by 
software  specialists  from  Government  laboratories  or 
specialists  from  some  other  advisary  organization.  These 
Air  Force  reviews  are  formally  documented  with  action  items 
assigned  to  the  Air  Force  or  contractor  for  resolution  by 
specified  dates. 

AFR  800-14  requires  at  least  four  formal  reviews;  the  systems 
requirements  review  (SRR),  the  system  design  review  (SDR), 
the  preliminary  design  review  (PDR),  and  the  critical  design 
review  (CDR).  The  PDR  and  CDR  were  described  earlier.  The 
SRR  is  conducted  after  a significant  portion  of  the  system 
functional  requirements  have  been  established  and  is  used 
to  evaluate  contractor  responsiveness  to  the  statement  of 
work  and  the  contractor's  interpretation  of  the  system 
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requirements.  The  SDR  is  conducted  prior  to  the  beginning  of 
preliminary  design  by  the  contractor  and  is  used  to  review 
system  documentation  and  assess  the  degree  of  accomplishment 
of  the  engineering  management  activities.  It  is  advisable 
that  the  contractor  arrange  informal  design  reviews  prior  to 
PDR  and  CDR  with  the  Air  Force  technical  specialists,  so 
that  minor  problems  can  be  ironed  out  in  advance  to  permit 
PDR  and  CDR  to  focus  on  the  most  important  matters.  The 
following  figure  illustrates  the  relationship  of  these 
reviews  to  the  software  development  process. 

Test  Plan 

Several  test  plans  are  prepared  during  the  software 
development  cycle  to  define  procedures  for  package  integra- 
tion and  test  and  system  integration  and  test.  These  plans 
explain  who  does  what  and  when.  They  may  also  specify 
test  requirements  down  to  the  unit  level.  The  principal 
test  plans  prepared  are  for  development  test  & evaluation 
(DT&E),  initial  operational  test  and  evaluation  (lOT&E) 
and  follow-on  operational  test  and  evaluation  (FOT&E). 

Note  that  the  contractor  and  the  Program  Office  conduct  DT&E 
testing,  while  the  Air  Force  Test  and  Evaluation  Center 
(AFTEC)  is  responsible  for  lOT&E  and  FOT&E  testing.  These 
test  plans  are  developed  from  data  provided  by  requirements, 
system  analysis,  package  design,  and  unit  design.  They  are 
prepared  to  support  the  Test  and  Evaluation  Master  Plan 
(TEMP)  which  is  the  overall  master  test  plan  prepared  in 
conjunction  with  the  PMD.  These  test  plans  are  used  to  define 
the  test  problems  to  be  solved  by  the  software  along  with 
acceptable  solutions.  R&M  test  criteria  are,  of  course, 
included. 

The  use  of  the  DT&E  test  plan  is  formally  evaluated  via 
preliminary  qualification  testing  (PQT)  and  formal  qualifi- 
cation testing  (FQT).  PQT  is  conducted  on  the  "critical" 
functions  of  the  software  package  during  the  time  period 
between  completion  of  CDR  and  the  start  of  FQT.  FQT  is  a 
complete  and  comprehensive  test  of  the  software  package 
performed  after  completion  of  the  design,  and  which  culminates 
in  a functional  configuration  audit  (FCA). 

Technical  Manuals 

While  the  various  specifications  and  design  documents 
described  above  document  the  exact  structure  of  the  soft- 
ware, those  documents  are  not  necessarily  suitable  for  field 
use  in  training  and  operations.  The  technical  manuals  are 
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written  using  those  specifications  and  documents,  but  are 
written  by  people  who  know  how  to  convey  that  information 
to  field  personnel  in  tne  most  effective  way.  The  manuals 
normally  include  the  following  types: 

* User's  Manual 

* Computer  Operator's  Manual 

* Software  Maintenance  Manual 

* System  Maintenance  Manual 

All  types  may  not  be  needed  for  a particular  system.  As 
mentioned  before,  the  contractor's  and  Air  Force  data 
systems  must  include  administrative  procedures  to  insure 
that  these  manuals  reflect  all  changes  in  specifications 
and  design  documents  throughout  the  software  life  cycle. 

RESOURCE  REQUIREMFNTS 

At  least  95%  of  the  software  engineering  work  described 
above  will  be  done  under  contract.  The  brief  comments  to 
follow  apply  to  the  other  5%  or  less  which  is  in-house  Air 
Force  work,  but  is  critical  to  program  success. 

Technical  Manpower 

The  Air  Force  Program  Manager  responsible  for  a system 
which  includes  a major  software  subsystem,  should  have  a 
software  specialist  assigned  to  his  staff.  At  the  very 
least,  he  should  make  arrangements  with  one  of  the  Govern- 
ment R&D  organizations  which  specialize  in  software  work 
to  provide  engineering  support  for  preparation  of  statements 
of  work,  requirements  and  specifications,  and  to  review 
design  and  test  results.  Software  groups  are  located  in  the 
Information  Sciences  Division  of  Rome  Air  Development  Center, 
in  the  Air  Force  Avionics  Laboratory,  and  in  the  Directorate 
of  Information  Systems  Technology  at  Electronic  Systems 
Division.  Of  course,  the  captive  corporations  such  as 
Aerospace  Corporation  and  MITRE  Corporation  have  software 
people,  and  other  corporate  groups  could  be  employed  as 
advise 's  assuming  no  conflict  of  commercial  interest  is 
present. 
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Reference  Publications 


There  are  numerous  commercial  books  and  journals  which 
describe  all  aspects  of  software  engineering.  The  Program 
Office  library  should  include  some  recent  publications  on 
the  implementation  of  modern  programming  practices  on 
large  systems,  such  as  the  RADC  Structure  Programming 
Series,  which  any  software  engineer  can  locate  through 
standard  reference  sources  such  as  the  National  Technical 
Information  Service. 

The  following  documents,  among  many  others,  would  be  useful: 

* Record,  1975  International  Conference  on 
Reliable  Software,  Los  Angeles  CA,  21-23  Apr 
75,  published  by  the  Institute  of  Electrical 
and  Electronic  Engineers,  Inc.,  345  East 
47th  St.,  New  York,  NY  10017. 

* Record,  Software  Quality  and  Assurance 
Workshop,  San  Diego  CA,  15-17  Nov  78, 
published  by  the  Association  for  Computing 
Machinery,  P.O.  Box  1205,  Church  Street 
Station,  New  York,  NY  10249. 

* "Structured  Programming  Series,"  IBM  Corp., 
RADC-TR-74-300,  Vols.  I - XV,  available 
through  the  National  Technical  Information 
Service  (NTIS),  5285  Port  Royal  Road, 
Springfield,  VA  22161. 

* "Software  Reliability  Study,"  TRW  Systems 
Group,  RADC-TR-76-238,  Aug  76,  also  available 
through  NTIS.  (A030798) 

* Record,  Second  Software  Life  Cycle  Management 
Workshop,  Atlanta,  GA,  21-22  Aug  78, 
available  through  IEEE. 

In  addition  the  Program  Office  library  should  include: 

* APR  800-14,  Vols.  I & II,  "Management  of 
Computer  Resources  in  Systems,"  Sep  75. 

* AFSCP  800-7,  "Configuration  Management," 

Dec  77. 

* AFSCR  74-1,  "Quality  Assurance  Program," 

Nov  78. 

Note:  RADC-TR-74-300,  Vols  I - VI  (A016771 )(A018046)(A013255)(A015794) 
(A003339)(A007796)  Vol  VII  & VII  Addendum  (A008639)(A016414) 

Vols  VIII  - XV  (A016415){A008640)(A008861)(AG16416)(A026947) 
(A020858)(A015795)(A016668) 
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* MIL-S-52779,  "Software  Quality  Assurance 
Program  Requirements." 

* MIL-STD-1521 , "Technical  Reviews  and  Audits 
for  Systems,  Equipment,  and  Computer 
Programs. " 

* MIL-STD-483,  "Configuration  Management 
Procedures  for  Systems,  Equipment, 
Munitions,  and  Computer  Programs." 


Facilities 

The  Air  Force  Program  Office  does  not  need  any  special 
facilities  for  software  engineering  management.  The  con- 
tractor, of  course,  will  need  access  to  data  processing 
facilities  adequate  to  handle  the  scope  of  his  project. 


SYNOPSIS 


Software  reliability  and  maintainability  have  begun  to 
develop  into  an  organized  body  of  knowledge  and  ideas  in 
only  the  past  several  years.  There  is  no  military  standard 
focused  on  this  area,  but  an  Air  Force  manual  is  being 
written  that  may  adequately  encompass  software  R&M.  Soft- 
ware costs  in  systems  development  are  snowballing,  because 
of  the  expanding  use  of  digital  technology  and  system  data 
processing  computers. 

The  most  critical  elements  in  development  of  a large  software 
system  to  achieve  adequate  R&M  are  the  use  of  logically 
developed  and  carefully  controlled  organization  for  both  the 
problems  being  solved  and  the  people  doing  the  computer 
programming,  the  use  of  a restrictive  set  of  programming 
rules  to  prevent  the  growth  of  a computation  logic  maze, 
development  of  thorough  test  procedures,  and  special  emphasis 
on  documentation  and  overall  program  management. 

The  Air  Force  Program  Office  needs  to  arrange  for  expert 
consultants  to  help  plan  and  monitor  software  development. 


APPENDIX  A 

RELIABILITY  AND  MAINTAINABILITY  TRAINING  SOURCES 

There  are  a variety  of  means  for  obtaining  R&M  training  ranging 
from  Master's  Degree  programs  to  lectures  sponsored  by  local  R&M 
professional  groups.  These  are  described  by  categories  below: 

MASTER'S  DEGREE  PROGRAMS 

Starting  September  1979,  the  Air  Force  Institute  of  Technology 
will  begin  a program  leading  to  an  MS  in  Electrical  Engineering, 

Reliability  option.  Further  information  can  be  obtained  from: 

AFIT/ENG 

ATTN:  Dr.  T.  Regul inski 

Wright-Patterson  AFB  OH  45433 

Another  program  leading  to  an  MS  degree  with  a Reliability 
Engineering  option  is  offered  by  the  University  of  Arizona.  Infor- 
mation may  be  obtained  from: 

Dr.  Dimitri  Kececioglu 

Aerospace  & Mechanical  Engineering  Department  p 

Building  #16  ! 

The  University  of  Arizona 

Tuscon,  Arizona  85721  ' 

SHORT  COURSES  IN  R&M  1 

AFIT  offers  two  short  courses  in  R&M.  Reliability  QMT  372  is  a 
15-day  course  designed  to  provide  an  understanding  of  R&M  principles 
and  basic  skills.  System  Reliability/Maintainability  QMT  576  is  a 
50-day  graduate  level  course  designed  to  provide  the  attendee  with  the 
ability  to  carry  out  the  functions  of  a reliability/maintainability 


engineer.  Information  on  these  may  be  obtained  from: 


AFIT/LS 

Wright-Patterson  AFB  OH  45433 

Other  courses  are  offered  by  various  sources.  The  R&M  trainee 
will  find  training  courses  listed  in  R&M  newsletters  and,  if  he  joins  an 
R&M  professional  society,  will  receive  many  course  announcements  by  direct 
direct  mail.  A listing  of  agencies  who  have  sponsored  recent  R&M  courses 
are  as  follows: 

The  Reliability  Analysis  Center 
RADC/RBRAC 

Griffiss  AFB  NY  13441 

(Sponsors  a four  day  training  course,  "Reliability  Design  Guidelines", 
held  in  various  locations  throughout  the  country). 

Arizona  State  University 

Reliability  Engineering  and  Management  Institute 
Dr.  Dimitri  Kececioglu,  Director 
Building  #16 

The  University  of  Arizona 

Tuscon,  Arizona  85721 

(Periodic  five-day  courses  at  Tuscon) 

Short  Course  Program  Office 

UCLA  Extension 

1D995  LeConte  Ave 

Los  Angeles  CA  90024 

(Various  five-day  R&M  courses  at  UCLA) 

ARINC  Research  Corporation 
2551  Riva  Road 
Annapolis  MO  21401 

(R8M  seminars  of  various  lengths  and  locations) 

University  of  Wisconsin  - Extension 
Dept  of  Engineering 
432  North  Lake  Street 
Madison  WI  53706 

Office  of  Business  Extension 
215  Business  Building 
Oklahoma  State  University 
Stillwater  OK  74074 
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University  College  of  Syracuse  University 
610  East  Fayette  Street 
Syracuse  NY  13202. 


Stat-a-Matrix  Institute 
P.O.  Box  2021 
Menlo  Park  Station 
Edison  NO  08817 

Continuing  Engineering  Education 
George  Washington  University 
Wash  DC  20052 

(Various  courses  in  Washington.  DC) 

TUSTIN  Institute  of  Technology 
22  E.  Los  Olivos  Street 
Santa  Barbara  CA  93105 

(Short  courses  on  vibration,  shock  and  noise  effects, 
measurement  and  design  guidance) 


R&M  SYMPOSIUMS 

The  Annual  Reliability  and  Maintainability  Symposium  is  a three-day 

conference  held  each  year  in  January,  alternating  in  location  between  the 

east  and  west  coasts.  It  is  jointly  sponsored  by  all  the  R&M  professional 

societies  and  its  technical  program  includes  tutorial  sessions  for  R&M 

trainees.  Every  paper  presented  (75  at  the  1979  R&M  Symposium)  is  printed 

in  the  proceedings.  A copy  is  given  to  each  attendee  and  the  IEEE 

Reliability  Society  also  sends  a copy  to  each  of  its  members.  Past 

conference  proceedings  are  available  at  $18.00  a copy  from: 

Order  Department 
IEEE  Service  Center 
445  Hoes  Lane 
Piscataway  NJ  08854 


Information  on  the  symposium  may  be  obtained  from  any  of  the  R&M 
professional  societies  listed  under  R&M  Professional  Societies  below. 

The  Institute  of  Environmental  Sciences  sponsors  seminars  on  R&M 
topics  such  as  testing,  and  offers  a variety  of  seminar  proceedings.  A 
listing  of  available  proceedings  and  prices,  as  well  as  Information  of 
future  seminars  may  be  obtained  from  the  Institute,  at  the  address 
listed  below  under  R&M  Professional  Societies. 

Other  annual  national  symposiums  include  the  Reliability  Physics 

Symposium  sponsored  by  the  IEEE  Reliability  Society,  and  the  Product 

Liability  Conference.  Information  on  the  latter  may  be  obtained  from; 

Richard  M.  Jacobs,  PE 
23  Rumson  Road 
Livingston  NJ  07039 

There  are  also  various  national  symposiums  on  R&M  specialty  areas, 
such  as  Software  Reliability,  sponsored  by  the  R&M  professional  societies. 
Industrial  concerns  and  Academic  Institutions  either  annually  or  on  an 
ad-hoc  basis. 

R&M  professional  societies  also  sponsor  local  symposiums,  such  as 
the  Annual  Spring  Reliability  Seminar  in  the  Boston  area,  and  local 
meetings  featuring  topics  of  current  interest.  One  need  not  belong  to 
the  sponsoring  society  to  attend.  The  chief  source  of  information  about 
these  affairs  are  the  various  society  newsletters  and  local  announcements. 
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PERIODICALS 


R&M  technical  papers  are  published  in  the  IEEE  Transactions  on 
Reliability,  which  also  serves  as  the  Journal  of  the  Electronics  Division 
of  the  American  Society  for  Quality  Control.  Copies  are  provided  to 
members  of  these  organizations  and  are  available  in  many  technical 
libraries.  Copies  may  also  be  purchased  from  the  IEEE. 

Evaluation  Engineering  Magazine  also  publishes  R&M  articles. 

Subscriptions  are  available  from: 

A.  Verner  Nelson  Associates 
The  Nelson  Building 
1282  Old  Skokie  Road 
Highland  Park  IL  60035 

The  RAC  Newsletter  provides  information  on  the  operations  the 
Reliability  Analysis  Center,  RADC  R&M  activities,  R&M  symposiums  and 
conferences,  and  general  items  of  R&M  interest.  Subscriptions  are  free 
from: 

RADC/RBRAC 

Griff iss  AFB  NY  13441 

Finally,  each  R&M  professional  society  provides  a newsletter  to  its 
members.  Local  chapters  often  provide  their  own  newsletter  listing 
all  local  conferences  of  interest. 

R&M  PROFESSIONAL  SOCIETIES 

As  indicated  above,  membership  in  an  R&M  society  can  be  a fruitful 
source  of  information.  The  major  R&M  professional  societies  are: 

The  IEEE  Reliability  Society 

institute  of  Electrical  and  Electronic  Engineers 

345  E.  47th  Street 

New  York  City,  NY  10017 
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The  Reliability  Division  and  the  Electronics  Division 
American  Society  for  Quality  Control 
161  West  Wisconsin  Ave 
Milwaukee  WI  53203 

The  Institute  of  Environmental  Sciences 
940  East  Northwest  Highway 
Mount  Prospect  IL  60056 

The  Society  of  Logistics  Engineers 
3322  South  Memorial  Parkway,  Suite  2 
Huntsville  AL  35801 

The  Society  of  Reliability  Engineers 

P.O.  Box  r31 

Crum  Lynne  PA  19022 

The  System  Safety  Society 
Box  A 

Newport  Beach  CA  92663 
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